[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786267941944493-20982-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 09:32:22 dist-test-slave-7kzw krb5kdc[20992](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 09:32:22 dist-test-slave-7kzw krb5kdc[20992](info): set up 2 sockets
Aug 09 09:32:22 dist-test-slave-7kzw krb5kdc[20992](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260809 09:32:23.325994 20982 mini_kdc.cc:121] Time spent starting KDC: real 1.344s user 0.004s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786267941944493-20982-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786267941944493-20982-0/krb5kdc/test-user.keytab.
Aug 09 09:32:23 dist-test-slave-7kzw krb5kdc[20992](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786267943, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T09:32:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T09:32:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.125.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.125.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.125.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.190.keytab.
Entry for principal kudu/127.20.125.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.190.keytab.
I20260809 09:32:23.458515 20982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
/tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786267941944493-20982-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786267941944493-20982-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786267941944493-20982-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786267941944493-20982-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.125.190:44807
--webserver_interface=127.20.125.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.190.keytab
--principal=kudu/127.20.125.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.125.148:40107
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.125.190:44807
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786267941944493-20982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786267941944493-20982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786267941944493-20982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 09:32:23.570533 21010 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 09:32:23.570801 21010 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 09:32:23.570840 21010 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 09:32:23.574744 21010 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 09:32:23.574798 21010 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 09:32:23.574817 21010 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 09:32:23.574833 21010 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 09:32:23.574848 21010 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 09:32:23.579888 21010 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.125.148:40107
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786267941944493-20982-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786267941944493-20982-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.125.190:44807
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.190.keytab
--principal=kudu/127.20.125.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.125.190:44807
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786267941944493-20982-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.125.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21010
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786267941944493-20982-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 08:38:55 UTC on ccycloud.quasar-njjleu.root.comops.site
build id 4575935
I20260809 09:32:23.580835 21010 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 09:32:23.581822 21010 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 09:32:23.590569 21018 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:32:23.590708 21017 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:32:23.590770 21010 server_base.cc:1034] running on GCE node
W20260809 09:32:23.590632 21020 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:32:23.591421 21010 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 09:32:23.592434 21010 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 09:32:23.593650 21010 hybrid_clock.cc:648] HybridClock initialized: now 1786267943593660 us; error 51 us; skew 500 ppm
Aug 09 09:32:23 dist-test-slave-7kzw krb5kdc[20992](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786267943, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 09:32:23.596894 21010 init.cc:377] Logged in from keytab as kudu/127.20.125.190@KRBTEST.COM (short username kudu)
I20260809 09:32:23.597757 21010 webserver.cc:460] Webserver started at http://127.20.125.190:40369/ using document root <none> and password file <none>
I20260809 09:32:23.598332 21010 fs_manager.cc:359] Metadata directory not provided
I20260809 09:32:23.598429 21010 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 09:32:23.598690 21010 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 09:32:23.600700 21010 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786267941944493-20982-0/minicluster-data/master-0/data/instance:
uuid: "7a075f58388a45bc9c76df3ea1163a76"
format_stamp: "Formatted at 2026-08-09 09:32:23 on dist-test-slave-7kzw"
server_key: "7a9475dbdc186dc00bca58f94329cf6f"
server_key_iv: "602cfc1dbfdbc6179b7e58ea37111754"
server_key_version: "encryptionkey@0"
I20260809 09:32:23.601276 21010 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786267941944493-20982-0/minicluster-data/master-0/wal/instance:
uuid: "7a075f58388a45bc9c76df3ea1163a76"
format_stamp: "Formatted at 2026-08-09 09:32:23 on dist-test-slave-7kzw"
server_key: "7a9475dbdc186dc00bca58f94329cf6f"
server_key_iv: "602cfc1dbfdbc6179b7e58ea37111754"
server_key_version: "encryptionkey@0"
I20260809 09:32:23.605082 21010 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.006s
I20260809 09:32:23.607616 21034 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 09:32:23.608893 21010 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260809 09:32:23.609048 21010 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786267941944493-20982-0/minicluster-data/master-0/data,/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786267941944493-20982-0/minicluster-data/master-0/wal
uuid: "7a075f58388a45bc9c76df3ea1163a76"
format_stamp: "Formatted at 2026-08-09 09:32:23 on dist-test-slave-7kzw"
server_key: "7a9475dbdc186dc00bca58f94329cf6f"
server_key_iv: "602cfc1dbfdbc6179b7e58ea37111754"
server_key_version: "encryptionkey@0"
I20260809 09:32:23.609184 21010 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786267941944493-20982-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786267941944493-20982-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786267941944493-20982-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 09:32:23.624503 21010 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 09:32:23.627655 21010 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 09:32:23.627858 21010 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 09:32:23.635076 21010 rpc_server.cc:307] RPC server started. Bound to: 127.20.125.190:44807
I20260809 09:32:23.635097 21108 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.125.190:44807 every 8 connection(s)
I20260809 09:32:23.636009 21010 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786267941944493-20982-0/minicluster-data/master-0/data/info.pb
I20260809 09:32:23.638850 21109 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 09:32:23.644371 21109 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7a075f58388a45bc9c76df3ea1163a76: Bootstrap starting.
I20260809 09:32:23.645885 20982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu as pid 21010
I20260809 09:32:23.646046 20982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786267941944493-20982-0/minicluster-data/master-0/wal/instance
I20260809 09:32:23.647485 21109 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7a075f58388a45bc9c76df3ea1163a76: Neither blocks nor log segments found. Creating new log.
I20260809 09:32:23.647845 20982 external_mini_cluster.cc:1428] Setting key 50be5ff1f63247ea21e072d36903e545
I20260809 09:32:23.648392 21109 log.cc:824] T 00000000000000000000000000000000 P 7a075f58388a45bc9c76df3ea1163a76: Log is configured to *not* fsync() on all Append() calls
I20260809 09:32:23.650743 21109 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7a075f58388a45bc9c76df3ea1163a76: No bootstrap required, opened a new log
I20260809 09:32:23.654803 21109 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7a075f58388a45bc9c76df3ea1163a76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a075f58388a45bc9c76df3ea1163a76" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 44807 } }
I20260809 09:32:23.655040 21109 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7a075f58388a45bc9c76df3ea1163a76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 09:32:23.655134 21109 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7a075f58388a45bc9c76df3ea1163a76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a075f58388a45bc9c76df3ea1163a76, State: Initialized, Role: FOLLOWER
I20260809 09:32:23.655622 21109 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7a075f58388a45bc9c76df3ea1163a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a075f58388a45bc9c76df3ea1163a76" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 44807 } }
I20260809 09:32:23.655790 21109 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7a075f58388a45bc9c76df3ea1163a76 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 09:32:23.655856 21109 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7a075f58388a45bc9c76df3ea1163a76 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 09:32:23.655952 21109 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7a075f58388a45bc9c76df3ea1163a76 [term 0 FOLLOWER]: Advancing to term 1
I20260809 09:32:23.657033 21109 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7a075f58388a45bc9c76df3ea1163a76 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a075f58388a45bc9c76df3ea1163a76" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 44807 } }
I20260809 09:32:23.657364 21109 leader_election.cc:302] T 00000000000000000000000000000000 P 7a075f58388a45bc9c76df3ea1163a76 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7a075f58388a45bc9c76df3ea1163a76; no voters:
I20260809 09:32:23.657681 21109 leader_election.cc:288] T 00000000000000000000000000000000 P 7a075f58388a45bc9c76df3ea1163a76 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 09:32:23.657739 21115 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7a075f58388a45bc9c76df3ea1163a76 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 09:32:23.657927 21115 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7a075f58388a45bc9c76df3ea1163a76 [term 1 LEADER]: Becoming Leader. State: Replica: 7a075f58388a45bc9c76df3ea1163a76, State: Running, Role: LEADER
I20260809 09:32:23.658218 21115 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7a075f58388a45bc9c76df3ea1163a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a075f58388a45bc9c76df3ea1163a76" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 44807 } }
I20260809 09:32:23.658928 21109 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7a075f58388a45bc9c76df3ea1163a76 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 09:32:23.659399 21115 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7a075f58388a45bc9c76df3ea1163a76 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7a075f58388a45bc9c76df3ea1163a76" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a075f58388a45bc9c76df3ea1163a76" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 44807 } } }
I20260809 09:32:23.659518 21115 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7a075f58388a45bc9c76df3ea1163a76 [sys.catalog]: This master's current role is: LEADER
I20260809 09:32:23.660143 21123 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 09:32:23.660899 21116 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7a075f58388a45bc9c76df3ea1163a76 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7a075f58388a45bc9c76df3ea1163a76. Latest consensus state: current_term: 1 leader_uuid: "7a075f58388a45bc9c76df3ea1163a76" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a075f58388a45bc9c76df3ea1163a76" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 44807 } } }
I20260809 09:32:23.661031 21116 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7a075f58388a45bc9c76df3ea1163a76 [sys.catalog]: This master's current role is: LEADER
I20260809 09:32:23.666649 21123 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 09 09:32:23 dist-test-slave-7kzw krb5kdc[20992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786267943, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.125.190@KRBTEST.COM
I20260809 09:32:23.675052 21123 catalog_manager.cc:1347] Generated new cluster ID: fc5a15bed374438da57797f1427e44e2
I20260809 09:32:23.675122 21123 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 09:32:23.680831 21112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:23.651277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46144 (local address 127.20.125.190:44807)
0809 09:32:23.651739 (+ 462us) server_negotiation.cc:206] Beginning negotiation
0809 09:32:23.651758 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0809 09:32:23.652970 (+ 1212us) server_negotiation.cc:407] Connection header received
0809 09:32:23.655690 (+ 2720us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:32:23.655708 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:32:23.656079 (+ 371us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:32:23.656446 (+ 367us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:32:23.659513 (+ 3067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:23.661513 (+ 2000us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:23.663417 (+ 1904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:23.664231 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:23.664366 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:23.670187 (+ 5821us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:32:23.670194 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:32:23.670211 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:32:23.670243 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:32:23.672721 (+ 2478us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:32:23.673469 (+ 748us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:32:23.673475 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:32:23.673481 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:32:23.673641 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:32:23.674091 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:32:23.674094 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:32:23.674096 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:32:23.674653 (+ 557us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:32:23.674878 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0809 09:32:23.679278 (+ 4400us) server_negotiation.cc:299] Negotiation successful
0809 09:32:23.679570 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":298,"thread_start_us":136,"threads_started":1}
I20260809 09:32:23.693184 21123 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 09:32:23.694324 21123 catalog_manager.cc:1504] Loading token signing keys...
I20260809 09:32:23.701702 21123 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7a075f58388a45bc9c76df3ea1163a76: Generated new TSK 0
I20260809 09:32:23.702397 21123 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.20.125.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.125.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.125.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.129.keytab.
Entry for principal kudu/127.20.125.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.129.keytab.
I20260809 09:32:23.737706 20982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
/tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786267941944493-20982-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786267941944493-20982-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786267941944493-20982-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786267941944493-20982-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.125.129:0
--local_ip_for_outbound_sockets=127.20.125.129
--webserver_interface=127.20.125.129
--webserver_port=0
--tserver_master_addrs=127.20.125.190:44807
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.129.keytab
--principal=kudu/127.20.125.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.125.148:40107
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786267941944493-20982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786267941944493-20982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786267941944493-20982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 09:32:23.862265 21148 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 09:32:23.862547 21148 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 09:32:23.862599 21148 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 09:32:23.866662 21148 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 09:32:23.866717 21148 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 09:32:23.866775 21148 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.125.129
I20260809 09:32:23.871819 21148 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.125.148:40107
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786267941944493-20982-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786267941944493-20982-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.129.keytab
--principal=kudu/127.20.125.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.125.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786267941944493-20982-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.125.129
--webserver_port=0
--tserver_master_addrs=127.20.125.190:44807
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21148
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.125.129
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786267941944493-20982-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 08:38:55 UTC on ccycloud.quasar-njjleu.root.comops.site
build id 4575935
I20260809 09:32:23.872776 21148 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 09:32:23.873728 21148 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 09:32:23.883735 21155 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:32:23.883783 21154 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:32:23.883769 21159 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:32:23.883870 21148 server_base.cc:1034] running on GCE node
I20260809 09:32:23.884339 21148 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 09:32:23.884958 21148 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 09:32:23.886150 21148 hybrid_clock.cc:648] HybridClock initialized: now 1786267943886119 us; error 44 us; skew 500 ppm
Aug 09 09:32:23 dist-test-slave-7kzw krb5kdc[20992](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786267943, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 09:32:23.889058 21148 init.cc:377] Logged in from keytab as kudu/127.20.125.129@KRBTEST.COM (short username kudu)
I20260809 09:32:23.889915 21148 webserver.cc:460] Webserver started at http://127.20.125.129:42217/ using document root <none> and password file <none>
I20260809 09:32:23.890538 21148 fs_manager.cc:359] Metadata directory not provided
I20260809 09:32:23.890630 21148 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 09:32:23.890867 21148 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 09:32:23.892886 21148 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786267941944493-20982-0/minicluster-data/ts-0/data/instance:
uuid: "9cdf4fd9b3a34e4cbc7bb38caaee7cb1"
format_stamp: "Formatted at 2026-08-09 09:32:23 on dist-test-slave-7kzw"
server_key: "464a3f28384ae6dcbc1005ab7ba45c68"
server_key_iv: "713ce6b92e6cfbcd16b85a9e17a8a4e8"
server_key_version: "encryptionkey@0"
I20260809 09:32:23.893468 21148 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786267941944493-20982-0/minicluster-data/ts-0/wal/instance:
uuid: "9cdf4fd9b3a34e4cbc7bb38caaee7cb1"
format_stamp: "Formatted at 2026-08-09 09:32:23 on dist-test-slave-7kzw"
server_key: "464a3f28384ae6dcbc1005ab7ba45c68"
server_key_iv: "713ce6b92e6cfbcd16b85a9e17a8a4e8"
server_key_version: "encryptionkey@0"
I20260809 09:32:23.897246 21148 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260809 09:32:23.899641 21172 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 09:32:23.900874 21148 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 09:32:23.901028 21148 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786267941944493-20982-0/minicluster-data/ts-0/data,/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786267941944493-20982-0/minicluster-data/ts-0/wal
uuid: "9cdf4fd9b3a34e4cbc7bb38caaee7cb1"
format_stamp: "Formatted at 2026-08-09 09:32:23 on dist-test-slave-7kzw"
server_key: "464a3f28384ae6dcbc1005ab7ba45c68"
server_key_iv: "713ce6b92e6cfbcd16b85a9e17a8a4e8"
server_key_version: "encryptionkey@0"
I20260809 09:32:23.901158 21148 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786267941944493-20982-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786267941944493-20982-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786267941944493-20982-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 09:32:23.915192 21148 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 09:32:23.918401 21148 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 09:32:23.918645 21148 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 09:32:23.919271 21148 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 09:32:23.920133 21148 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 09:32:23.920217 21148 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 09:32:23.920291 21148 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 09:32:23.920336 21148 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 09:32:23.929831 21148 rpc_server.cc:307] RPC server started. Bound to: 127.20.125.129:41107
I20260809 09:32:23.929867 21332 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.125.129:41107 every 8 connection(s)
I20260809 09:32:23.930800 21148 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786267941944493-20982-0/minicluster-data/ts-0/data/info.pb
I20260809 09:32:23.935189 20982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu as pid 21148
I20260809 09:32:23.935312 20982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786267941944493-20982-0/minicluster-data/ts-0/wal/instance
I20260809 09:32:23.935590 20982 external_mini_cluster.cc:1428] Setting key 6c6015021260ccf6963a2f81518e7642
Aug 09 09:32:23 dist-test-slave-7kzw krb5kdc[20992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786267943, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.129@KRBTEST.COM for kudu/127.20.125.190@KRBTEST.COM
I20260809 09:32:23.946735 21112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:23.932938 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.129:56943 (local address 127.20.125.190:44807)
0809 09:32:23.933107 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0809 09:32:23.933111 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 09:32:23.934083 (+ 972us) server_negotiation.cc:407] Connection header received
0809 09:32:23.935067 (+ 984us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:32:23.935071 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:32:23.935144 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:32:23.935265 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:32:23.936809 (+ 1544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:23.937671 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:23.938968 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:23.939455 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:23.939584 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:23.942863 (+ 3279us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:32:23.942867 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:32:23.942870 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:32:23.942896 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:32:23.944535 (+ 1639us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:32:23.945134 (+ 599us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:32:23.945137 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:32:23.945139 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:32:23.945221 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:32:23.945574 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:32:23.945577 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:32:23.945579 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:32:23.945773 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:32:23.945886 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0809 09:32:23.946397 (+ 511us) server_negotiation.cc:299] Negotiation successful
0809 09:32:23.946556 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
WARNING: no policy specified for kudu/127.20.125.130@KRBTEST.COM; defaulting to no policy
I20260809 09:32:23.947568 21337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:23.933268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:44807 (local address 127.20.125.129:56943)
0809 09:32:23.933918 (+ 650us) negotiation.cc:107] Waiting for socket to connect
0809 09:32:23.933959 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 09:32:23.934841 (+ 882us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:32:23.935455 (+ 614us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:32:23.935464 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:32:23.935932 (+ 468us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:32:23.936598 (+ 666us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:32:23.936618 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:32:23.937841 (+ 1223us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:32:23.937845 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:32:23.938827 (+ 982us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:32:23.938837 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:32:23.939718 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:32:23.939725 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:32:23.939958 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:23.940699 (+ 741us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 09:32:23.940725 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:32:23.942671 (+ 1946us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 09:32:23.944696 (+ 2025us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:32:23.944702 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:32:23.944718 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:32:23.945002 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:32:23.945316 (+ 314us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:32:23.945320 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:32:23.945322 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:32:23.945467 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:32:23.945885 (+ 418us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 09:32:23.945893 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 09:32:23.946139 (+ 246us) client_negotiation.cc:769] Sending connection context
0809 09:32:23.946337 (+ 198us) client_negotiation.cc:240] Negotiation successful
0809 09:32:23.946697 (+ 360us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":469,"thread_start_us":114,"threads_started":1}
I20260809 09:32:23.948938 21335 heartbeater.cc:347] Connected to a master server at 127.20.125.190:44807
I20260809 09:32:23.949254 21335 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.125.130@KRBTEST.COM" created.
I20260809 09:32:23.949842 21335 heartbeater.cc:511] Master 127.20.125.190:44807 requested a full tablet report, sending...
I20260809 09:32:23.951642 21059 ts_manager.cc:194] Registered new tserver with Master: 9cdf4fd9b3a34e4cbc7bb38caaee7cb1 (127.20.125.129:41107)
I20260809 09:32:23.952848 21059 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.125.129@KRBTEST.COM'} at 127.20.125.129:56943
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.125.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.130.keytab.
Entry for principal kudu/127.20.125.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.130.keytab.
I20260809 09:32:23.965565 20982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
/tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786267941944493-20982-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786267941944493-20982-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786267941944493-20982-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786267941944493-20982-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.125.130:0
--local_ip_for_outbound_sockets=127.20.125.130
--webserver_interface=127.20.125.130
--webserver_port=0
--tserver_master_addrs=127.20.125.190:44807
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.130.keytab
--principal=kudu/127.20.125.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.125.148:40107
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786267941944493-20982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786267941944493-20982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786267941944493-20982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 09:32:24.080230 21340 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 09:32:24.080471 21340 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 09:32:24.080514 21340 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 09:32:24.084498 21340 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 09:32:24.084549 21340 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 09:32:24.084647 21340 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.125.130
I20260809 09:32:24.089694 21340 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.125.148:40107
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786267941944493-20982-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786267941944493-20982-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.130.keytab
--principal=kudu/127.20.125.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.125.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786267941944493-20982-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.125.130
--webserver_port=0
--tserver_master_addrs=127.20.125.190:44807
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21340
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.125.130
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786267941944493-20982-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 08:38:55 UTC on ccycloud.quasar-njjleu.root.comops.site
build id 4575935
I20260809 09:32:24.090708 21340 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 09:32:24.091753 21340 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 09:32:24.101364 21345 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:32:24.101359 21348 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:32:24.101785 21340 server_base.cc:1034] running on GCE node
W20260809 09:32:24.101994 21346 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:32:24.102537 21340 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 09:32:24.103349 21340 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 09:32:24.104540 21340 hybrid_clock.cc:648] HybridClock initialized: now 1786267944104457 us; error 97 us; skew 500 ppm
Aug 09 09:32:24 dist-test-slave-7kzw krb5kdc[20992](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786267944, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 09:32:24.111662 21340 init.cc:377] Logged in from keytab as kudu/127.20.125.130@KRBTEST.COM (short username kudu)
I20260809 09:32:24.112651 21340 webserver.cc:460] Webserver started at http://127.20.125.130:45825/ using document root <none> and password file <none>
I20260809 09:32:24.113300 21340 fs_manager.cc:359] Metadata directory not provided
I20260809 09:32:24.113401 21340 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 09:32:24.113664 21340 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 09:32:24.115863 21340 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786267941944493-20982-0/minicluster-data/ts-1/data/instance:
uuid: "5ccd426ab7074f4289520147bdcbda09"
format_stamp: "Formatted at 2026-08-09 09:32:24 on dist-test-slave-7kzw"
server_key: "e0a4879933e706ced4f93f800709a167"
server_key_iv: "fa31b492202dea867c9abc499a350e41"
server_key_version: "encryptionkey@0"
I20260809 09:32:24.116474 21340 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786267941944493-20982-0/minicluster-data/ts-1/wal/instance:
uuid: "5ccd426ab7074f4289520147bdcbda09"
format_stamp: "Formatted at 2026-08-09 09:32:24 on dist-test-slave-7kzw"
server_key: "e0a4879933e706ced4f93f800709a167"
server_key_iv: "fa31b492202dea867c9abc499a350e41"
server_key_version: "encryptionkey@0"
I20260809 09:32:24.120280 21340 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 09:32:24.123090 21361 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 09:32:24.124437 21340 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 09:32:24.124586 21340 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786267941944493-20982-0/minicluster-data/ts-1/data,/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786267941944493-20982-0/minicluster-data/ts-1/wal
uuid: "5ccd426ab7074f4289520147bdcbda09"
format_stamp: "Formatted at 2026-08-09 09:32:24 on dist-test-slave-7kzw"
server_key: "e0a4879933e706ced4f93f800709a167"
server_key_iv: "fa31b492202dea867c9abc499a350e41"
server_key_version: "encryptionkey@0"
I20260809 09:32:24.124706 21340 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786267941944493-20982-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786267941944493-20982-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786267941944493-20982-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 09:32:24.150887 21340 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 09:32:24.154428 21340 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 09:32:24.154696 21340 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 09:32:24.155344 21340 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 09:32:24.156234 21340 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 09:32:24.156308 21340 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 09:32:24.156383 21340 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 09:32:24.156428 21340 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 09:32:24.166428 21340 rpc_server.cc:307] RPC server started. Bound to: 127.20.125.130:33355
I20260809 09:32:24.166447 21523 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.125.130:33355 every 8 connection(s)
I20260809 09:32:24.167402 21340 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786267941944493-20982-0/minicluster-data/ts-1/data/info.pb
I20260809 09:32:24.173039 20982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu as pid 21340
I20260809 09:32:24.173135 20982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786267941944493-20982-0/minicluster-data/ts-1/wal/instance
I20260809 09:32:24.173416 20982 external_mini_cluster.cc:1428] Setting key ca8eadb319cd2ce4fed315aa2d238b4d
Aug 09 09:32:24 dist-test-slave-7kzw krb5kdc[20992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786267944, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.130@KRBTEST.COM for kudu/127.20.125.190@KRBTEST.COM
I20260809 09:32:24.182853 21112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:24.169468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.130:45889 (local address 127.20.125.190:44807)
0809 09:32:24.169768 (+ 300us) server_negotiation.cc:206] Beginning negotiation
0809 09:32:24.169773 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 09:32:24.170894 (+ 1121us) server_negotiation.cc:407] Connection header received
0809 09:32:24.171797 (+ 903us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:32:24.171799 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:32:24.171845 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:32:24.171920 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:32:24.173236 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:24.173876 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:24.175353 (+ 1477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:24.175570 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:24.175659 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:24.178614 (+ 2955us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:32:24.178621 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:32:24.178627 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:32:24.178660 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:32:24.180539 (+ 1879us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:32:24.181095 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:32:24.181102 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:32:24.181106 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:32:24.181208 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:32:24.181558 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:32:24.181564 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:32:24.181566 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:32:24.181865 (+ 299us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:32:24.181987 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0809 09:32:24.182503 (+ 516us) server_negotiation.cc:299] Negotiation successful
0809 09:32:24.182648 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158}
I20260809 09:32:24.183578 21528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:24.169975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:44807 (local address 127.20.125.130:45889)
0809 09:32:24.170742 (+ 767us) negotiation.cc:107] Waiting for socket to connect
0809 09:32:24.170794 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0809 09:32:24.171611 (+ 817us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:32:24.172116 (+ 505us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:32:24.172123 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:32:24.172506 (+ 383us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:32:24.173026 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:32:24.173043 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:32:24.174029 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:32:24.174032 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:32:24.175226 (+ 1194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:32:24.175239 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:32:24.175734 (+ 495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:32:24.175740 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:32:24.175970 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:24.176641 (+ 671us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 09:32:24.176668 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:32:24.178379 (+ 1711us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 09:32:24.180667 (+ 2288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:32:24.180673 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:32:24.180715 (+ 42us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:32:24.180968 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:32:24.181307 (+ 339us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:32:24.181310 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:32:24.181311 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:32:24.181443 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:32:24.181978 (+ 535us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 09:32:24.181983 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 09:32:24.182213 (+ 230us) client_negotiation.cc:769] Sending connection context
0809 09:32:24.182367 (+ 154us) client_negotiation.cc:240] Negotiation successful
0809 09:32:24.182679 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":567,"thread_start_us":114,"threads_started":1}
I20260809 09:32:24.184921 21526 heartbeater.cc:347] Connected to a master server at 127.20.125.190:44807
WARNING: no policy specified for kudu/127.20.125.131@KRBTEST.COM; defaulting to no policy
I20260809 09:32:24.185221 21526 heartbeater.cc:464] Registering TS with master...
I20260809 09:32:24.185726 21526 heartbeater.cc:511] Master 127.20.125.190:44807 requested a full tablet report, sending...
I20260809 09:32:24.186847 21059 ts_manager.cc:194] Registered new tserver with Master: 5ccd426ab7074f4289520147bdcbda09 (127.20.125.130:33355)
I20260809 09:32:24.187593 21059 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.125.130@KRBTEST.COM'} at 127.20.125.130:45889
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.125.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.125.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.131.keytab.
Entry for principal kudu/127.20.125.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.131.keytab.
I20260809 09:32:24.205814 20982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
/tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786267941944493-20982-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786267941944493-20982-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786267941944493-20982-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786267941944493-20982-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.125.131:0
--local_ip_for_outbound_sockets=127.20.125.131
--webserver_interface=127.20.125.131
--webserver_port=0
--tserver_master_addrs=127.20.125.190:44807
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.131.keytab
--principal=kudu/127.20.125.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.125.148:40107
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786267941944493-20982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786267941944493-20982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786267941944493-20982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 09:32:24.317876 21533 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 09:32:24.318126 21533 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 09:32:24.318171 21533 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 09:32:24.322129 21533 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 09:32:24.322187 21533 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 09:32:24.322244 21533 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.125.131
I20260809 09:32:24.327533 21533 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.125.148:40107
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786267941944493-20982-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786267941944493-20982-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.131.keytab
--principal=kudu/127.20.125.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.125.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786267941944493-20982-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.125.131
--webserver_port=0
--tserver_master_addrs=127.20.125.190:44807
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21533
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.125.131
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786267941944493-20982-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 08:38:55 UTC on ccycloud.quasar-njjleu.root.comops.site
build id 4575935
I20260809 09:32:24.328670 21533 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 09:32:24.329701 21533 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 09:32:24.338989 21543 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:32:24.338965 21547 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:32:24.338973 21540 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:32:24.339054 21533 server_base.cc:1034] running on GCE node
I20260809 09:32:24.339597 21533 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 09:32:24.340251 21533 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 09:32:24.341437 21533 hybrid_clock.cc:648] HybridClock initialized: now 1786267944341409 us; error 51 us; skew 500 ppm
Aug 09 09:32:24 dist-test-slave-7kzw krb5kdc[20992](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786267944, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 09:32:24.344210 21533 init.cc:377] Logged in from keytab as kudu/127.20.125.131@KRBTEST.COM (short username kudu)
I20260809 09:32:24.345018 21533 webserver.cc:460] Webserver started at http://127.20.125.131:37375/ using document root <none> and password file <none>
I20260809 09:32:24.345556 21533 fs_manager.cc:359] Metadata directory not provided
I20260809 09:32:24.345650 21533 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 09:32:24.345901 21533 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 09:32:24.347903 21533 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786267941944493-20982-0/minicluster-data/ts-2/data/instance:
uuid: "023255c91b6b4907b62a9d5db6caaba1"
format_stamp: "Formatted at 2026-08-09 09:32:24 on dist-test-slave-7kzw"
server_key: "5a596f8c53094fabc570736693ecc938"
server_key_iv: "acd23d9c516eb2e13685faa03688b4a6"
server_key_version: "encryptionkey@0"
I20260809 09:32:24.348475 21533 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786267941944493-20982-0/minicluster-data/ts-2/wal/instance:
uuid: "023255c91b6b4907b62a9d5db6caaba1"
format_stamp: "Formatted at 2026-08-09 09:32:24 on dist-test-slave-7kzw"
server_key: "5a596f8c53094fabc570736693ecc938"
server_key_iv: "acd23d9c516eb2e13685faa03688b4a6"
server_key_version: "encryptionkey@0"
I20260809 09:32:24.352165 21533 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.003s
I20260809 09:32:24.354589 21558 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 09:32:24.355826 21533 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260809 09:32:24.355970 21533 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786267941944493-20982-0/minicluster-data/ts-2/data,/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786267941944493-20982-0/minicluster-data/ts-2/wal
uuid: "023255c91b6b4907b62a9d5db6caaba1"
format_stamp: "Formatted at 2026-08-09 09:32:24 on dist-test-slave-7kzw"
server_key: "5a596f8c53094fabc570736693ecc938"
server_key_iv: "acd23d9c516eb2e13685faa03688b4a6"
server_key_version: "encryptionkey@0"
I20260809 09:32:24.356082 21533 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786267941944493-20982-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786267941944493-20982-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786267941944493-20982-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 09:32:24.382895 21533 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 09:32:24.386123 21533 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 09:32:24.386336 21533 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 09:32:24.386988 21533 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 09:32:24.387995 21533 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 09:32:24.388067 21533 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 09:32:24.388141 21533 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 09:32:24.388197 21533 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 09:32:24.397790 21533 rpc_server.cc:307] RPC server started. Bound to: 127.20.125.131:35657
I20260809 09:32:24.397838 21732 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.125.131:35657 every 8 connection(s)
I20260809 09:32:24.398757 21533 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786267941944493-20982-0/minicluster-data/ts-2/data/info.pb
I20260809 09:32:24.402735 20982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu as pid 21533
I20260809 09:32:24.402858 20982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786267941944493-20982-0/minicluster-data/ts-2/wal/instance
I20260809 09:32:24.403137 20982 external_mini_cluster.cc:1428] Setting key 707345a679236581ef5a594cb9c6e312
Aug 09 09:32:24 dist-test-slave-7kzw krb5kdc[20992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786267944, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.131@KRBTEST.COM for kudu/127.20.125.190@KRBTEST.COM
I20260809 09:32:24.412847 21112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:24.400764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.131:39479 (local address 127.20.125.190:44807)
0809 09:32:24.400914 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0809 09:32:24.400917 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 09:32:24.401719 (+ 802us) server_negotiation.cc:407] Connection header received
0809 09:32:24.403330 (+ 1611us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:32:24.403333 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:32:24.403382 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:32:24.403458 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:32:24.404957 (+ 1499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:24.405595 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:24.406878 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:24.407088 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:24.407178 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:24.409676 (+ 2498us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:32:24.409680 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:32:24.409683 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:32:24.409710 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:32:24.411064 (+ 1354us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:32:24.411520 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:32:24.411523 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:32:24.411524 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:32:24.411598 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:32:24.411895 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:32:24.411897 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:32:24.411898 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:32:24.412063 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:32:24.412139 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 09:32:24.412597 (+ 458us) server_negotiation.cc:299] Negotiation successful
0809 09:32:24.412729 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260809 09:32:24.413625 21736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:24.401145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:44807 (local address 127.20.125.131:39479)
0809 09:32:24.401568 (+ 423us) negotiation.cc:107] Waiting for socket to connect
0809 09:32:24.401611 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 09:32:24.403080 (+ 1469us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:32:24.403643 (+ 563us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:32:24.403653 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:32:24.404129 (+ 476us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:32:24.404755 (+ 626us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:32:24.404811 (+ 56us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:32:24.405721 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:32:24.405726 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:32:24.406764 (+ 1038us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:32:24.406774 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:32:24.407228 (+ 454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:32:24.407231 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:32:24.407385 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:24.407989 (+ 604us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 09:32:24.408005 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:32:24.409518 (+ 1513us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 09:32:24.411169 (+ 1651us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:32:24.411173 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:32:24.411186 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:32:24.411433 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:32:24.411678 (+ 245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:32:24.411685 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:32:24.411687 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:32:24.411814 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:32:24.412150 (+ 336us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 09:32:24.412154 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 09:32:24.412365 (+ 211us) client_negotiation.cc:769] Sending connection context
0809 09:32:24.412515 (+ 150us) client_negotiation.cc:240] Negotiation successful
0809 09:32:24.412840 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":247,"thread_start_us":104,"threads_started":1}
I20260809 09:32:24.414788 21734 heartbeater.cc:347] Connected to a master server at 127.20.125.190:44807
I20260809 09:32:24.415045 21734 heartbeater.cc:464] Registering TS with master...
I20260809 09:32:24.415518 21734 heartbeater.cc:511] Master 127.20.125.190:44807 requested a full tablet report, sending...
I20260809 09:32:24.416832 21058 ts_manager.cc:194] Registered new tserver with Master: 023255c91b6b4907b62a9d5db6caaba1 (127.20.125.131:35657)
I20260809 09:32:24.417593 21058 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.125.131@KRBTEST.COM'} at 127.20.125.131:39479
I20260809 09:32:24.417691 20982 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 09 09:32:24 dist-test-slave-7kzw krb5kdc[20992](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786267944, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 09:32:24 dist-test-slave-7kzw krb5kdc[20992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786267944, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.20.125.129@KRBTEST.COM
I20260809 09:32:24.449463 21746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:24.439245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48794 (local address 127.20.125.129:41107)
0809 09:32:24.439630 (+ 385us) server_negotiation.cc:206] Beginning negotiation
0809 09:32:24.439649 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0809 09:32:24.439859 (+ 210us) server_negotiation.cc:407] Connection header received
0809 09:32:24.440050 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:32:24.440056 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:32:24.440262 (+ 206us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:32:24.440557 (+ 295us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:32:24.441269 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:24.442044 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:24.442957 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:24.443333 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:24.443461 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:24.445952 (+ 2491us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:32:24.445961 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:32:24.445974 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:32:24.445998 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:32:24.447599 (+ 1601us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:32:24.448145 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:32:24.448151 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:32:24.448152 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:32:24.448247 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:32:24.448517 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:32:24.448520 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:32:24.448521 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:32:24.448805 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:32:24.448977 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0809 09:32:24.449180 (+ 203us) server_negotiation.cc:299] Negotiation successful
0809 09:32:24.449315 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":269,"thread_start_us":161,"threads_started":1}
W20260809 09:32:24.450449 21212 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48794
Aug 09 09:32:24 dist-test-slave-7kzw krb5kdc[20992](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786267944, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 09:32:24 dist-test-slave-7kzw krb5kdc[20992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786267944, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.125.129@KRBTEST.COM
I20260809 09:32:24.481287 21746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:24.472113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48804 (local address 127.20.125.129:41107)
0809 09:32:24.472289 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0809 09:32:24.472294 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 09:32:24.472432 (+ 138us) server_negotiation.cc:407] Connection header received
0809 09:32:24.472602 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:32:24.472605 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:32:24.472657 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:32:24.472780 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:32:24.473501 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:24.474011 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:24.474916 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:24.475140 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:24.475256 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:24.477797 (+ 2541us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:32:24.477800 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:32:24.477802 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:32:24.477828 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:32:24.479503 (+ 1675us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:32:24.480040 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:32:24.480043 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:32:24.480044 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:32:24.480144 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:32:24.480438 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:32:24.480441 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:32:24.480442 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:32:24.480608 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:32:24.480728 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0809 09:32:24.480960 (+ 232us) server_negotiation.cc:299] Negotiation successful
0809 09:32:24.481063 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260809 09:32:24.482445 20982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu with pid 21148
I20260809 09:32:24.495045 20982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu with pid 21340
I20260809 09:32:24.516117 20982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu with pid 21533
I20260809 09:32:24.531270 20982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu with pid 21010
2026-08-09T09:32:24Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (2585 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786267941944493-20982-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 09:32:24 dist-test-slave-7kzw krb5kdc[21757](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 09:32:24 dist-test-slave-7kzw krb5kdc[21757](info): set up 2 sockets
Aug 09 09:32:24 dist-test-slave-7kzw krb5kdc[21757](info): commencing operation
krb5kdc: starting...
W20260809 09:32:25.813066 20982 mini_kdc.cc:121] Time spent starting KDC: real 1.255s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786267941944493-20982-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786267941944493-20982-0/krb5kdc/test-user.keytab.
Aug 09 09:32:25 dist-test-slave-7kzw krb5kdc[21757](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786267945, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T09:32:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T09:32:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.125.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.125.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.125.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.190.keytab.
Entry for principal kudu/127.20.125.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.190.keytab.
I20260809 09:32:25.950251 20982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
/tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786267941944493-20982-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786267941944493-20982-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786267941944493-20982-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786267941944493-20982-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.125.190:44085
--webserver_interface=127.20.125.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.190.keytab
--principal=kudu/127.20.125.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.125.148:44089
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.125.190:44085
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786267941944493-20982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786267941944493-20982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786267941944493-20982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 09:32:26.067263 21773 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 09:32:26.067624 21773 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 09:32:26.067699 21773 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 09:32:26.071707 21773 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 09:32:26.071812 21773 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 09:32:26.071848 21773 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 09:32:26.071888 21773 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 09:32:26.071918 21773 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 09:32:26.077262 21773 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.125.148:44089
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786267941944493-20982-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786267941944493-20982-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.125.190:44085
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.190.keytab
--principal=kudu/127.20.125.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.125.190:44085
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786267941944493-20982-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.125.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21773
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786267941944493-20982-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 08:38:55 UTC on ccycloud.quasar-njjleu.root.comops.site
build id 4575935
I20260809 09:32:26.078428 21773 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 09:32:26.079561 21773 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 09:32:26.088546 21782 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:32:26.088553 21780 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:32:26.088570 21786 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:32:26.089035 21773 server_base.cc:1034] running on GCE node
I20260809 09:32:26.089653 21773 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 09:32:26.090828 21773 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 09:32:26.092031 21773 hybrid_clock.cc:648] HybridClock initialized: now 1786267946092002 us; error 42 us; skew 500 ppm
Aug 09 09:32:26 dist-test-slave-7kzw krb5kdc[21757](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786267946, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 09:32:26.095036 21773 init.cc:377] Logged in from keytab as kudu/127.20.125.190@KRBTEST.COM (short username kudu)
I20260809 09:32:26.095990 21773 webserver.cc:460] Webserver started at http://127.20.125.190:45763/ using document root <none> and password file <none>
I20260809 09:32:26.096575 21773 fs_manager.cc:359] Metadata directory not provided
I20260809 09:32:26.096640 21773 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 09:32:26.096911 21773 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 09:32:26.099094 21773 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786267941944493-20982-0/minicluster-data/master-0/data/instance:
uuid: "68ea3752701b4f9ebddd0510da15e734"
format_stamp: "Formatted at 2026-08-09 09:32:26 on dist-test-slave-7kzw"
server_key: "3f2e206f89d18e8dc7170d336a091550"
server_key_iv: "4dd0e74eca91a13c13b2827f08cb4e63"
server_key_version: "encryptionkey@0"
I20260809 09:32:26.099679 21773 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786267941944493-20982-0/minicluster-data/master-0/wal/instance:
uuid: "68ea3752701b4f9ebddd0510da15e734"
format_stamp: "Formatted at 2026-08-09 09:32:26 on dist-test-slave-7kzw"
server_key: "3f2e206f89d18e8dc7170d336a091550"
server_key_iv: "4dd0e74eca91a13c13b2827f08cb4e63"
server_key_version: "encryptionkey@0"
I20260809 09:32:26.103498 21773 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260809 09:32:26.105892 21793 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 09:32:26.107333 21773 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 09:32:26.107481 21773 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786267941944493-20982-0/minicluster-data/master-0/data,/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786267941944493-20982-0/minicluster-data/master-0/wal
uuid: "68ea3752701b4f9ebddd0510da15e734"
format_stamp: "Formatted at 2026-08-09 09:32:26 on dist-test-slave-7kzw"
server_key: "3f2e206f89d18e8dc7170d336a091550"
server_key_iv: "4dd0e74eca91a13c13b2827f08cb4e63"
server_key_version: "encryptionkey@0"
I20260809 09:32:26.107599 21773 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786267941944493-20982-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786267941944493-20982-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786267941944493-20982-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 09:32:26.135242 21773 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 09:32:26.138892 21773 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 09:32:26.139113 21773 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 09:32:26.146806 21773 rpc_server.cc:307] RPC server started. Bound to: 127.20.125.190:44085
I20260809 09:32:26.146831 21873 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.125.190:44085 every 8 connection(s)
I20260809 09:32:26.147928 21773 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786267941944493-20982-0/minicluster-data/master-0/data/info.pb
I20260809 09:32:26.150779 21874 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 09:32:26.156281 21874 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 68ea3752701b4f9ebddd0510da15e734: Bootstrap starting.
I20260809 09:32:26.158034 20982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu as pid 21773
I20260809 09:32:26.158154 20982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786267941944493-20982-0/minicluster-data/master-0/wal/instance
I20260809 09:32:26.158445 20982 external_mini_cluster.cc:1428] Setting key 15040a45a3fba4a7ed3d271940233f7a
I20260809 09:32:26.159024 21874 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 68ea3752701b4f9ebddd0510da15e734: Neither blocks nor log segments found. Creating new log.
I20260809 09:32:26.159881 21874 log.cc:824] T 00000000000000000000000000000000 P 68ea3752701b4f9ebddd0510da15e734: Log is configured to *not* fsync() on all Append() calls
I20260809 09:32:26.162114 21874 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 68ea3752701b4f9ebddd0510da15e734: No bootstrap required, opened a new log
I20260809 09:32:26.166325 21874 raft_consensus.cc:348] T 00000000000000000000000000000000 P 68ea3752701b4f9ebddd0510da15e734 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68ea3752701b4f9ebddd0510da15e734" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 44085 } }
I20260809 09:32:26.166648 21874 raft_consensus.cc:374] T 00000000000000000000000000000000 P 68ea3752701b4f9ebddd0510da15e734 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 09:32:26.166709 21874 raft_consensus.cc:729] T 00000000000000000000000000000000 P 68ea3752701b4f9ebddd0510da15e734 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68ea3752701b4f9ebddd0510da15e734, State: Initialized, Role: FOLLOWER
I20260809 09:32:26.167232 21874 consensus_queue.cc:260] T 00000000000000000000000000000000 P 68ea3752701b4f9ebddd0510da15e734 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68ea3752701b4f9ebddd0510da15e734" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 44085 } }
I20260809 09:32:26.167457 21874 raft_consensus.cc:388] T 00000000000000000000000000000000 P 68ea3752701b4f9ebddd0510da15e734 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 09:32:26.167536 21874 raft_consensus.cc:482] T 00000000000000000000000000000000 P 68ea3752701b4f9ebddd0510da15e734 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 09:32:26.167650 21874 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 68ea3752701b4f9ebddd0510da15e734 [term 0 FOLLOWER]: Advancing to term 1
Aug 09 09:32:26 dist-test-slave-7kzw krb5kdc[21757](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786267945, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.125.190@KRBTEST.COM
I20260809 09:32:26.168622 21874 raft_consensus.cc:504] T 00000000000000000000000000000000 P 68ea3752701b4f9ebddd0510da15e734 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68ea3752701b4f9ebddd0510da15e734" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 44085 } }
I20260809 09:32:26.168998 21874 leader_election.cc:302] T 00000000000000000000000000000000 P 68ea3752701b4f9ebddd0510da15e734 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 68ea3752701b4f9ebddd0510da15e734; no voters:
I20260809 09:32:26.169374 21874 leader_election.cc:288] T 00000000000000000000000000000000 P 68ea3752701b4f9ebddd0510da15e734 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 09:32:26.169735 21884 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 68ea3752701b4f9ebddd0510da15e734 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 09:32:26.170255 21884 raft_consensus.cc:686] T 00000000000000000000000000000000 P 68ea3752701b4f9ebddd0510da15e734 [term 1 LEADER]: Becoming Leader. State: Replica: 68ea3752701b4f9ebddd0510da15e734, State: Running, Role: LEADER
I20260809 09:32:26.170660 21884 consensus_queue.cc:237] T 00000000000000000000000000000000 P 68ea3752701b4f9ebddd0510da15e734 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68ea3752701b4f9ebddd0510da15e734" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 44085 } }
I20260809 09:32:26.170812 21874 sys_catalog.cc:564] T 00000000000000000000000000000000 P 68ea3752701b4f9ebddd0510da15e734 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 09:32:26.172412 21885 sys_catalog.cc:455] T 00000000000000000000000000000000 P 68ea3752701b4f9ebddd0510da15e734 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "68ea3752701b4f9ebddd0510da15e734" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68ea3752701b4f9ebddd0510da15e734" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 44085 } } }
I20260809 09:32:26.172554 21885 sys_catalog.cc:458] T 00000000000000000000000000000000 P 68ea3752701b4f9ebddd0510da15e734 [sys.catalog]: This master's current role is: LEADER
I20260809 09:32:26.172962 21887 sys_catalog.cc:455] T 00000000000000000000000000000000 P 68ea3752701b4f9ebddd0510da15e734 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 68ea3752701b4f9ebddd0510da15e734. Latest consensus state: current_term: 1 leader_uuid: "68ea3752701b4f9ebddd0510da15e734" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68ea3752701b4f9ebddd0510da15e734" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 44085 } } }
I20260809 09:32:26.173144 21887 sys_catalog.cc:458] T 00000000000000000000000000000000 P 68ea3752701b4f9ebddd0510da15e734 [sys.catalog]: This master's current role is: LEADER
I20260809 09:32:26.174938 21881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:26.159746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48600 (local address 127.20.125.190:44085)
0809 09:32:26.160222 (+ 476us) server_negotiation.cc:206] Beginning negotiation
0809 09:32:26.160241 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0809 09:32:26.160293 (+ 52us) server_negotiation.cc:407] Connection header received
0809 09:32:26.161125 (+ 832us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:32:26.161145 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:32:26.161462 (+ 317us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:32:26.161748 (+ 286us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:32:26.162767 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:26.163729 (+ 962us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:26.164997 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:26.165476 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:26.165612 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:26.168647 (+ 3035us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:32:26.168652 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:32:26.168667 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:32:26.168694 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:32:26.171178 (+ 2484us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:32:26.171753 (+ 575us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:32:26.171759 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:32:26.171765 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:32:26.171940 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:32:26.172271 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:32:26.172277 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:32:26.172279 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:32:26.172815 (+ 536us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:32:26.173028 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0809 09:32:26.173653 (+ 625us) server_negotiation.cc:299] Negotiation successful
0809 09:32:26.174004 (+ 351us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":319,"thread_start_us":149,"threads_started":1}
W20260809 09:32:26.177284 21907 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 68ea3752701b4f9ebddd0510da15e734: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 09:32:26.177371 21907 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 09:32:26.177433 21900 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 09:32:26.178200 21900 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 09:32:26.183872 21900 catalog_manager.cc:1347] Generated new cluster ID: 7f51a8762ebd4c6e820c1171fe5ccea5
I20260809 09:32:26.183952 21900 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 09:32:26.192773 21900 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 09:32:26.193714 21900 catalog_manager.cc:1504] Loading token signing keys...
I20260809 09:32:26.201889 21900 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 68ea3752701b4f9ebddd0510da15e734: Generated new TSK 0
I20260809 09:32:26.202804 21900 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.20.125.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.125.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.125.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.129.keytab.
Entry for principal kudu/127.20.125.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.129.keytab.
I20260809 09:32:26.242837 20982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
/tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786267941944493-20982-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786267941944493-20982-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786267941944493-20982-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786267941944493-20982-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.125.129:0
--local_ip_for_outbound_sockets=127.20.125.129
--webserver_interface=127.20.125.129
--webserver_port=0
--tserver_master_addrs=127.20.125.190:44085
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.129.keytab
--principal=kudu/127.20.125.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.125.148:44089
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786267941944493-20982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786267941944493-20982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786267941944493-20982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 09:32:26.377921 21915 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 09:32:26.378221 21915 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 09:32:26.378271 21915 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 09:32:26.382563 21915 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 09:32:26.382833 21915 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 09:32:26.382939 21915 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.125.129
I20260809 09:32:26.389143 21915 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.125.148:44089
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786267941944493-20982-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786267941944493-20982-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.129.keytab
--principal=kudu/127.20.125.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.125.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786267941944493-20982-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.125.129
--webserver_port=0
--tserver_master_addrs=127.20.125.190:44085
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21915
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.125.129
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786267941944493-20982-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 08:38:55 UTC on ccycloud.quasar-njjleu.root.comops.site
build id 4575935
I20260809 09:32:26.390779 21915 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 09:32:26.391969 21915 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 09:32:26.402961 21925 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:32:26.403110 21927 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:32:26.403249 21915 server_base.cc:1034] running on GCE node
W20260809 09:32:26.403097 21923 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:32:26.403833 21915 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 09:32:26.404556 21915 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 09:32:26.405757 21915 hybrid_clock.cc:648] HybridClock initialized: now 1786267946405738 us; error 30 us; skew 500 ppm
Aug 09 09:32:26 dist-test-slave-7kzw krb5kdc[21757](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786267946, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 09:32:26.408996 21915 init.cc:377] Logged in from keytab as kudu/127.20.125.129@KRBTEST.COM (short username kudu)
I20260809 09:32:26.409976 21915 webserver.cc:460] Webserver started at http://127.20.125.129:39803/ using document root <none> and password file <none>
I20260809 09:32:26.410615 21915 fs_manager.cc:359] Metadata directory not provided
I20260809 09:32:26.410727 21915 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 09:32:26.410983 21915 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 09:32:26.413213 21915 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786267941944493-20982-0/minicluster-data/ts-0/data/instance:
uuid: "24aba902e2154a19af6a54cd0f06274f"
format_stamp: "Formatted at 2026-08-09 09:32:26 on dist-test-slave-7kzw"
server_key: "f1c590409c9404293bfed75117b93a16"
server_key_iv: "5ff44d47ff5d34acfe6cf17b3d88bed1"
server_key_version: "encryptionkey@0"
I20260809 09:32:26.413818 21915 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786267941944493-20982-0/minicluster-data/ts-0/wal/instance:
uuid: "24aba902e2154a19af6a54cd0f06274f"
format_stamp: "Formatted at 2026-08-09 09:32:26 on dist-test-slave-7kzw"
server_key: "f1c590409c9404293bfed75117b93a16"
server_key_iv: "5ff44d47ff5d34acfe6cf17b3d88bed1"
server_key_version: "encryptionkey@0"
I20260809 09:32:26.417914 21915 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260809 09:32:26.420523 21937 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 09:32:26.421993 21915 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 09:32:26.422158 21915 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786267941944493-20982-0/minicluster-data/ts-0/data,/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786267941944493-20982-0/minicluster-data/ts-0/wal
uuid: "24aba902e2154a19af6a54cd0f06274f"
format_stamp: "Formatted at 2026-08-09 09:32:26 on dist-test-slave-7kzw"
server_key: "f1c590409c9404293bfed75117b93a16"
server_key_iv: "5ff44d47ff5d34acfe6cf17b3d88bed1"
server_key_version: "encryptionkey@0"
I20260809 09:32:26.422294 21915 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786267941944493-20982-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786267941944493-20982-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786267941944493-20982-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 09:32:26.433105 21915 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 09:32:26.436779 21915 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 09:32:26.437032 21915 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 09:32:26.437762 21915 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 09:32:26.438838 21915 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 09:32:26.438920 21915 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 09:32:26.439004 21915 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 09:32:26.439062 21915 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 09:32:26.449802 21915 rpc_server.cc:307] RPC server started. Bound to: 127.20.125.129:38863
I20260809 09:32:26.449833 22107 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.125.129:38863 every 8 connection(s)
I20260809 09:32:26.450907 21915 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786267941944493-20982-0/minicluster-data/ts-0/data/info.pb
I20260809 09:32:26.451591 20982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu as pid 21915
I20260809 09:32:26.451719 20982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786267941944493-20982-0/minicluster-data/ts-0/wal/instance
I20260809 09:32:26.452039 20982 external_mini_cluster.cc:1428] Setting key dbefba6ab6be2e0311d4fd7b3d93103c
Aug 09 09:32:26 dist-test-slave-7kzw krb5kdc[21757](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786267946, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.129@KRBTEST.COM for kudu/127.20.125.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.20.125.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.125.130@KRBTEST.COM" created.
I20260809 09:32:26.467769 21881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:26.453033 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.129:34025 (local address 127.20.125.190:44085)
0809 09:32:26.453207 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0809 09:32:26.453212 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 09:32:26.454505 (+ 1293us) server_negotiation.cc:407] Connection header received
0809 09:32:26.455782 (+ 1277us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:32:26.455785 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:32:26.455846 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:32:26.455941 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:32:26.457353 (+ 1412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:26.457966 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:26.459346 (+ 1380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:26.459618 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:26.459756 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:26.462856 (+ 3100us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:32:26.462860 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:32:26.462863 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:32:26.462888 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:32:26.464537 (+ 1649us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:32:26.465392 (+ 855us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:32:26.465396 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:32:26.465397 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:32:26.465483 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:32:26.465972 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:32:26.465975 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:32:26.465976 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:32:26.466214 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:32:26.466373 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0809 09:32:26.467260 (+ 887us) server_negotiation.cc:299] Negotiation successful
0809 09:32:26.467441 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260809 09:32:26.468626 22111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:26.453566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:44085 (local address 127.20.125.129:34025)
0809 09:32:26.454251 (+ 685us) negotiation.cc:107] Waiting for socket to connect
0809 09:32:26.454340 (+ 89us) client_negotiation.cc:174] Beginning negotiation
0809 09:32:26.455543 (+ 1203us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:32:26.456160 (+ 617us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:32:26.456168 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:32:26.456573 (+ 405us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:32:26.457184 (+ 611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:32:26.457200 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:32:26.458112 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:32:26.458116 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:32:26.459199 (+ 1083us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:32:26.459210 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:32:26.459803 (+ 593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:32:26.459808 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:32:26.459999 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:26.460699 (+ 700us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 09:32:26.460718 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:32:26.462656 (+ 1938us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 09:32:26.464899 (+ 2243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:32:26.464906 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:32:26.464925 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:32:26.465261 (+ 336us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:32:26.465617 (+ 356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:32:26.465621 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:32:26.465623 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:32:26.465829 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:32:26.466415 (+ 586us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 09:32:26.466424 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 09:32:26.466882 (+ 458us) client_negotiation.cc:769] Sending connection context
0809 09:32:26.467127 (+ 245us) client_negotiation.cc:240] Negotiation successful
0809 09:32:26.467462 (+ 335us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":387,"thread_start_us":196,"threads_started":1}
I20260809 09:32:26.470204 22108 heartbeater.cc:347] Connected to a master server at 127.20.125.190:44085
I20260809 09:32:26.470570 22108 heartbeater.cc:464] Registering TS with master...
I20260809 09:32:26.471272 22108 heartbeater.cc:511] Master 127.20.125.190:44085 requested a full tablet report, sending...
I20260809 09:32:26.473309 21821 ts_manager.cc:194] Registered new tserver with Master: 24aba902e2154a19af6a54cd0f06274f (127.20.125.129:38863)
I20260809 09:32:26.474633 21821 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.125.129@KRBTEST.COM'} at 127.20.125.129:34025
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.125.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.130.keytab.
Entry for principal kudu/127.20.125.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.130.keytab.
I20260809 09:32:26.486922 20982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
/tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786267941944493-20982-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786267941944493-20982-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786267941944493-20982-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786267941944493-20982-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.125.130:0
--local_ip_for_outbound_sockets=127.20.125.130
--webserver_interface=127.20.125.130
--webserver_port=0
--tserver_master_addrs=127.20.125.190:44085
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.130.keytab
--principal=kudu/127.20.125.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.125.148:44089
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786267941944493-20982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786267941944493-20982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786267941944493-20982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 09:32:26.604008 22116 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 09:32:26.604344 22116 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 09:32:26.604444 22116 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 09:32:26.608667 22116 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 09:32:26.608758 22116 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 09:32:26.608846 22116 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.125.130
I20260809 09:32:26.614387 22116 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.125.148:44089
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786267941944493-20982-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786267941944493-20982-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.130.keytab
--principal=kudu/127.20.125.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.125.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786267941944493-20982-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.125.130
--webserver_port=0
--tserver_master_addrs=127.20.125.190:44085
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22116
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.125.130
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786267941944493-20982-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 08:38:55 UTC on ccycloud.quasar-njjleu.root.comops.site
build id 4575935
I20260809 09:32:26.615577 22116 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 09:32:26.617106 22116 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 09:32:26.626904 22128 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:32:26.626930 22124 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:32:26.626978 22125 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:32:26.627187 22116 server_base.cc:1034] running on GCE node
I20260809 09:32:26.627708 22116 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 09:32:26.628374 22116 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 09:32:26.629591 22116 hybrid_clock.cc:648] HybridClock initialized: now 1786267946629573 us; error 27 us; skew 500 ppm
Aug 09 09:32:26 dist-test-slave-7kzw krb5kdc[21757](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786267946, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 09:32:26.632704 22116 init.cc:377] Logged in from keytab as kudu/127.20.125.130@KRBTEST.COM (short username kudu)
I20260809 09:32:26.633632 22116 webserver.cc:460] Webserver started at http://127.20.125.130:43185/ using document root <none> and password file <none>
I20260809 09:32:26.634277 22116 fs_manager.cc:359] Metadata directory not provided
I20260809 09:32:26.634373 22116 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 09:32:26.634644 22116 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 09:32:26.636724 22116 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786267941944493-20982-0/minicluster-data/ts-1/data/instance:
uuid: "49c5f666f9e14333b8cbdba292a944a7"
format_stamp: "Formatted at 2026-08-09 09:32:26 on dist-test-slave-7kzw"
server_key: "c08c7787f24aeb37904b0068e80f5c1c"
server_key_iv: "fbccaf2f398bce3ee07dabf228047543"
server_key_version: "encryptionkey@0"
I20260809 09:32:26.637315 22116 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786267941944493-20982-0/minicluster-data/ts-1/wal/instance:
uuid: "49c5f666f9e14333b8cbdba292a944a7"
format_stamp: "Formatted at 2026-08-09 09:32:26 on dist-test-slave-7kzw"
server_key: "c08c7787f24aeb37904b0068e80f5c1c"
server_key_iv: "fbccaf2f398bce3ee07dabf228047543"
server_key_version: "encryptionkey@0"
I20260809 09:32:26.641156 22116 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260809 09:32:26.643926 22137 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 09:32:26.645391 22116 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 09:32:26.645499 22116 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786267941944493-20982-0/minicluster-data/ts-1/data,/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786267941944493-20982-0/minicluster-data/ts-1/wal
uuid: "49c5f666f9e14333b8cbdba292a944a7"
format_stamp: "Formatted at 2026-08-09 09:32:26 on dist-test-slave-7kzw"
server_key: "c08c7787f24aeb37904b0068e80f5c1c"
server_key_iv: "fbccaf2f398bce3ee07dabf228047543"
server_key_version: "encryptionkey@0"
I20260809 09:32:26.645596 22116 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786267941944493-20982-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786267941944493-20982-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786267941944493-20982-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 09:32:26.656242 22116 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 09:32:26.659648 22116 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 09:32:26.659811 22116 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 09:32:26.660353 22116 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 09:32:26.661230 22116 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 09:32:26.661290 22116 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 09:32:26.661334 22116 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 09:32:26.661355 22116 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 09:32:26.670276 22116 rpc_server.cc:307] RPC server started. Bound to: 127.20.125.130:39051
I20260809 09:32:26.670372 22324 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.125.130:39051 every 8 connection(s)
I20260809 09:32:26.671377 22116 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786267941944493-20982-0/minicluster-data/ts-1/data/info.pb
I20260809 09:32:26.675060 20982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu as pid 22116
I20260809 09:32:26.675175 20982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786267941944493-20982-0/minicluster-data/ts-1/wal/instance
I20260809 09:32:26.675453 20982 external_mini_cluster.cc:1428] Setting key eaa65dadd860c11dba612a42c2257636
Aug 09 09:32:26 dist-test-slave-7kzw krb5kdc[21757](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786267946, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.130@KRBTEST.COM for kudu/127.20.125.190@KRBTEST.COM
I20260809 09:32:26.685370 21881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:26.673544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.130:49065 (local address 127.20.125.190:44085)
0809 09:32:26.673703 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0809 09:32:26.673708 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 09:32:26.674628 (+ 920us) server_negotiation.cc:407] Connection header received
0809 09:32:26.675650 (+ 1022us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:32:26.675653 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:32:26.675715 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:32:26.675794 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:32:26.677123 (+ 1329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:26.677649 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:26.678932 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:26.679150 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:26.679230 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:26.681805 (+ 2575us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:32:26.681809 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:32:26.681812 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:32:26.681841 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:32:26.683342 (+ 1501us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:32:26.683857 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:32:26.683862 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:32:26.683864 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:32:26.683947 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:32:26.684289 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:32:26.684294 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:32:26.684296 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:32:26.684477 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:32:26.684583 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0809 09:32:26.685060 (+ 477us) server_negotiation.cc:299] Negotiation successful
0809 09:32:26.685182 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260809 09:32:26.686122 22330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:26.673977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:44085 (local address 127.20.125.130:49065)
0809 09:32:26.674421 (+ 444us) negotiation.cc:107] Waiting for socket to connect
0809 09:32:26.674512 (+ 91us) client_negotiation.cc:174] Beginning negotiation
0809 09:32:26.675412 (+ 900us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:32:26.675960 (+ 548us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:32:26.675967 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:32:26.676438 (+ 471us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:32:26.676981 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:32:26.676994 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:32:26.677777 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:32:26.677786 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:32:26.678769 (+ 983us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:32:26.678784 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:32:26.679299 (+ 515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:32:26.679305 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:32:26.679525 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:26.680160 (+ 635us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 09:32:26.680184 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:32:26.681636 (+ 1452us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 09:32:26.683462 (+ 1826us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:32:26.683468 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:32:26.683482 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:32:26.683752 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:32:26.684040 (+ 288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:32:26.684044 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:32:26.684046 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:32:26.684187 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:32:26.684582 (+ 395us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 09:32:26.684589 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 09:32:26.684827 (+ 238us) client_negotiation.cc:769] Sending connection context
0809 09:32:26.685016 (+ 189us) client_negotiation.cc:240] Negotiation successful
0809 09:32:26.685328 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":104,"threads_started":1}
WARNING: no policy specified for kudu/127.20.125.131@KRBTEST.COM; defaulting to no policy
I20260809 09:32:26.687352 22326 heartbeater.cc:347] Connected to a master server at 127.20.125.190:44085
I20260809 09:32:26.687760 22326 heartbeater.cc:464] Registering TS with master...
I20260809 09:32:26.688206 22326 heartbeater.cc:511] Master 127.20.125.190:44085 requested a full tablet report, sending...
I20260809 09:32:26.689343 21821 ts_manager.cc:194] Registered new tserver with Master: 49c5f666f9e14333b8cbdba292a944a7 (127.20.125.130:39051)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.125.131@KRBTEST.COM" created.
I20260809 09:32:26.690060 21821 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.125.130@KRBTEST.COM'} at 127.20.125.130:49065
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.125.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.131.keytab.
Entry for principal kudu/127.20.125.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.131.keytab.
I20260809 09:32:26.710271 20982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
/tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786267941944493-20982-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786267941944493-20982-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786267941944493-20982-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786267941944493-20982-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.125.131:0
--local_ip_for_outbound_sockets=127.20.125.131
--webserver_interface=127.20.125.131
--webserver_port=0
--tserver_master_addrs=127.20.125.190:44085
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.131.keytab
--principal=kudu/127.20.125.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.125.148:44089
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786267941944493-20982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786267941944493-20982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786267941944493-20982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 09:32:26.827899 22333 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 09:32:26.828171 22333 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 09:32:26.828214 22333 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 09:32:26.832446 22333 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 09:32:26.832516 22333 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 09:32:26.832577 22333 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.125.131
I20260809 09:32:26.838162 22333 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.125.148:44089
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786267941944493-20982-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786267941944493-20982-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.131.keytab
--principal=kudu/127.20.125.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.125.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786267941944493-20982-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.125.131
--webserver_port=0
--tserver_master_addrs=127.20.125.190:44085
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22333
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.125.131
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786267941944493-20982-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 08:38:55 UTC on ccycloud.quasar-njjleu.root.comops.site
build id 4575935
I20260809 09:32:26.839323 22333 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 09:32:26.840411 22333 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 09:32:26.851328 22345 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:32:26.851306 22340 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:32:26.851306 22342 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:32:26.852123 22333 server_base.cc:1034] running on GCE node
I20260809 09:32:26.852826 22333 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 09:32:26.853579 22333 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 09:32:26.854782 22333 hybrid_clock.cc:648] HybridClock initialized: now 1786267946854745 us; error 48 us; skew 500 ppm
Aug 09 09:32:26 dist-test-slave-7kzw krb5kdc[21757](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786267946, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 09:32:26.858083 22333 init.cc:377] Logged in from keytab as kudu/127.20.125.131@KRBTEST.COM (short username kudu)
I20260809 09:32:26.859288 22333 webserver.cc:460] Webserver started at http://127.20.125.131:36501/ using document root <none> and password file <none>
I20260809 09:32:26.859881 22333 fs_manager.cc:359] Metadata directory not provided
I20260809 09:32:26.859948 22333 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 09:32:26.860128 22333 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 09:32:26.862286 22333 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786267941944493-20982-0/minicluster-data/ts-2/data/instance:
uuid: "4acd715b6e064d6db562024ce1bb0632"
format_stamp: "Formatted at 2026-08-09 09:32:26 on dist-test-slave-7kzw"
server_key: "1c9c57fec13fb7eec153df8958a1ab62"
server_key_iv: "2746735bfed202bf55e7dee112fc5be8"
server_key_version: "encryptionkey@0"
I20260809 09:32:26.862947 22333 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786267941944493-20982-0/minicluster-data/ts-2/wal/instance:
uuid: "4acd715b6e064d6db562024ce1bb0632"
format_stamp: "Formatted at 2026-08-09 09:32:26 on dist-test-slave-7kzw"
server_key: "1c9c57fec13fb7eec153df8958a1ab62"
server_key_iv: "2746735bfed202bf55e7dee112fc5be8"
server_key_version: "encryptionkey@0"
I20260809 09:32:26.866952 22333 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260809 09:32:26.869591 22353 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 09:32:26.871292 22333 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 09:32:26.871407 22333 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786267941944493-20982-0/minicluster-data/ts-2/data,/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786267941944493-20982-0/minicluster-data/ts-2/wal
uuid: "4acd715b6e064d6db562024ce1bb0632"
format_stamp: "Formatted at 2026-08-09 09:32:26 on dist-test-slave-7kzw"
server_key: "1c9c57fec13fb7eec153df8958a1ab62"
server_key_iv: "2746735bfed202bf55e7dee112fc5be8"
server_key_version: "encryptionkey@0"
I20260809 09:32:26.871507 22333 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786267941944493-20982-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786267941944493-20982-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786267941944493-20982-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 09:32:26.888839 22333 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 09:32:26.893321 22333 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 09:32:26.893512 22333 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 09:32:26.894147 22333 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 09:32:26.895108 22333 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 09:32:26.895162 22333 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 09:32:26.895205 22333 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 09:32:26.895221 22333 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 09:32:26.905390 22333 rpc_server.cc:307] RPC server started. Bound to: 127.20.125.131:38453
I20260809 09:32:26.905416 22533 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.125.131:38453 every 8 connection(s)
I20260809 09:32:26.906438 22333 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786267941944493-20982-0/minicluster-data/ts-2/data/info.pb
I20260809 09:32:26.909047 20982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu as pid 22333
I20260809 09:32:26.909299 20982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786267941944493-20982-0/minicluster-data/ts-2/wal/instance
I20260809 09:32:26.909829 20982 external_mini_cluster.cc:1428] Setting key 36b67dd4eb159dc4eb79f5a3728b8148
Aug 09 09:32:26 dist-test-slave-7kzw krb5kdc[21757](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786267946, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.131@KRBTEST.COM for kudu/127.20.125.190@KRBTEST.COM
I20260809 09:32:26.923213 21881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:26.908958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.131:48339 (local address 127.20.125.190:44085)
0809 09:32:26.909145 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0809 09:32:26.909150 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 09:32:26.910021 (+ 871us) server_negotiation.cc:407] Connection header received
0809 09:32:26.911654 (+ 1633us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:32:26.911657 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:32:26.911716 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:32:26.911823 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:32:26.913222 (+ 1399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:26.913775 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:26.915122 (+ 1347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:26.915357 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:26.915476 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:26.918142 (+ 2666us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:32:26.918145 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:32:26.918147 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:32:26.918182 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:32:26.920061 (+ 1879us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:32:26.920693 (+ 632us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:32:26.920699 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:32:26.920705 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:32:26.920791 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:32:26.921167 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:32:26.921171 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:32:26.921173 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:32:26.921464 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:32:26.921586 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0809 09:32:26.922895 (+ 1309us) server_negotiation.cc:299] Negotiation successful
0809 09:32:26.923055 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260809 09:32:26.924501 22538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:26.909273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:44085 (local address 127.20.125.131:48339)
0809 09:32:26.909849 (+ 576us) negotiation.cc:107] Waiting for socket to connect
0809 09:32:26.909906 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0809 09:32:26.911429 (+ 1523us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:32:26.912068 (+ 639us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:32:26.912076 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:32:26.912534 (+ 458us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:32:26.913056 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:32:26.913070 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:32:26.913920 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:32:26.913926 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:32:26.914990 (+ 1064us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:32:26.915001 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:32:26.915525 (+ 524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:32:26.915529 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:32:26.915726 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:26.916317 (+ 591us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 09:32:26.916335 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:32:26.917951 (+ 1616us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 09:32:26.920233 (+ 2282us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:32:26.920239 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:32:26.920258 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:32:26.920566 (+ 308us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:32:26.920893 (+ 327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:32:26.920896 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:32:26.920898 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:32:26.921062 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:32:26.921582 (+ 520us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 09:32:26.921590 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 09:32:26.922267 (+ 677us) client_negotiation.cc:769] Sending connection context
0809 09:32:26.922822 (+ 555us) client_negotiation.cc:240] Negotiation successful
0809 09:32:26.923371 (+ 549us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":316,"thread_start_us":145,"threads_started":1}
I20260809 09:32:26.926199 22535 heartbeater.cc:347] Connected to a master server at 127.20.125.190:44085
I20260809 09:32:26.926659 22535 heartbeater.cc:464] Registering TS with master...
I20260809 09:32:26.927302 22535 heartbeater.cc:511] Master 127.20.125.190:44085 requested a full tablet report, sending...
I20260809 09:32:26.928470 21821 ts_manager.cc:194] Registered new tserver with Master: 4acd715b6e064d6db562024ce1bb0632 (127.20.125.131:38453)
I20260809 09:32:26.929093 21821 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.125.131@KRBTEST.COM'} at 127.20.125.131:48339
I20260809 09:32:26.937702 20982 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 09:32:26.951293 21881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:26.941833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48616 (local address 127.20.125.190:44085)
0809 09:32:26.941990 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0809 09:32:26.941994 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 09:32:26.942362 (+ 368us) server_negotiation.cc:407] Connection header received
0809 09:32:26.942594 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:32:26.942597 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:32:26.942648 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:32:26.942729 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:32:26.943681 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:26.944232 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:26.945389 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:26.945607 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:26.945698 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:26.947313 (+ 1615us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:32:26.947316 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:32:26.947318 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:32:26.947345 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:32:26.948998 (+ 1653us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:32:26.949641 (+ 643us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:32:26.949644 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:32:26.949645 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:32:26.949739 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:32:26.950108 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:32:26.950111 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:32:26.950112 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:32:26.950635 (+ 523us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:32:26.950772 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0809 09:32:26.951023 (+ 251us) server_negotiation.cc:299] Negotiation successful
0809 09:32:26.951137 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260809 09:32:26.957145 21821 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48616:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 09:32:26.960938 21821 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 09:32:26.977778 22550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:26.971156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.130:39051 (local address 127.0.0.1:42940)
0809 09:32:26.971598 (+ 442us) negotiation.cc:107] Waiting for socket to connect
0809 09:32:26.971657 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0809 09:32:26.972020 (+ 363us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:32:26.972819 (+ 799us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:32:26.972825 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:32:26.972851 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 09:32:26.973017 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:32:26.973026 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:32:26.974830 (+ 1804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:32:26.974834 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:32:26.976232 (+ 1398us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:32:26.976241 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:32:26.977229 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:32:26.977232 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:32:26.977335 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:26.977370 (+ 35us) client_negotiation.cc:769] Sending connection context
0809 09:32:26.977489 (+ 119us) client_negotiation.cc:240] Negotiation successful
0809 09:32:26.977615 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":239,"mutex_wait_us":62,"thread_start_us":131,"threads_started":1}
I20260809 09:32:26.977787 22551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:26.971283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42940 (local address 127.20.125.130:39051)
0809 09:32:26.972103 (+ 820us) server_negotiation.cc:206] Beginning negotiation
0809 09:32:26.972118 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 09:32:26.972147 (+ 29us) server_negotiation.cc:407] Connection header received
0809 09:32:26.972230 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:32:26.972236 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:32:26.972450 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:32:26.973196 (+ 746us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 09:32:26.973369 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:26.974628 (+ 1259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:26.976361 (+ 1733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:26.977102 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:26.977242 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:26.977382 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0809 09:32:26.977518 (+ 136us) server_negotiation.cc:299] Negotiation successful
0809 09:32:26.977636 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":662,"thread_start_us":126,"threads_started":1}
I20260809 09:32:26.978147 22553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:26.971405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.131:38453 (local address 127.0.0.1:51610)
0809 09:32:26.972325 (+ 920us) negotiation.cc:107] Waiting for socket to connect
0809 09:32:26.972344 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 09:32:26.972604 (+ 260us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:32:26.973309 (+ 705us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:32:26.973312 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:32:26.973333 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 09:32:26.973450 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:32:26.973459 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:32:26.974872 (+ 1413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:32:26.974876 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:32:26.976354 (+ 1478us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:32:26.976360 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:32:26.977595 (+ 1235us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:32:26.977599 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:32:26.977761 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:26.977857 (+ 96us) client_negotiation.cc:769] Sending connection context
0809 09:32:26.977927 (+ 70us) client_negotiation.cc:240] Negotiation successful
0809 09:32:26.977999 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":848,"mutex_wait_us":164,"thread_start_us":123,"threads_started":1}
I20260809 09:32:26.978236 22557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:26.972560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48966 (local address 127.20.125.129:38863)
0809 09:32:26.973166 (+ 606us) server_negotiation.cc:206] Beginning negotiation
0809 09:32:26.973172 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 09:32:26.973391 (+ 219us) server_negotiation.cc:407] Connection header received
0809 09:32:26.973626 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:32:26.973631 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:32:26.973781 (+ 150us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:32:26.973928 (+ 147us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 09:32:26.974114 (+ 186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:26.975050 (+ 936us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:26.976482 (+ 1432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:26.977296 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:26.977402 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:26.977525 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0809 09:32:26.977994 (+ 469us) server_negotiation.cc:299] Negotiation successful
0809 09:32:26.978092 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":372,"thread_start_us":124,"threads_started":1}
I20260809 09:32:26.978628 22554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:26.971961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.129:38863 (local address 127.0.0.1:48966)
0809 09:32:26.973298 (+ 1337us) negotiation.cc:107] Waiting for socket to connect
0809 09:32:26.973312 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 09:32:26.973461 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:32:26.973909 (+ 448us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:32:26.973912 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:32:26.973929 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 09:32:26.974010 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:32:26.974018 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:32:26.975174 (+ 1156us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:32:26.975177 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:32:26.976328 (+ 1151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:32:26.976340 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:32:26.977627 (+ 1287us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:32:26.977630 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:32:26.977784 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:26.977854 (+ 70us) client_negotiation.cc:769] Sending connection context
0809 09:32:26.978386 (+ 532us) client_negotiation.cc:240] Negotiation successful
0809 09:32:26.978489 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1278,"mutex_wait_us":73,"thread_start_us":96,"threads_started":1}
I20260809 09:32:26.978865 22555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:26.972164 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51610 (local address 127.20.125.131:38453)
0809 09:32:26.972740 (+ 576us) server_negotiation.cc:206] Beginning negotiation
0809 09:32:26.972754 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 09:32:26.972776 (+ 22us) server_negotiation.cc:407] Connection header received
0809 09:32:26.972854 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:32:26.972858 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:32:26.973115 (+ 257us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:32:26.973328 (+ 213us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 09:32:26.973619 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:26.974757 (+ 1138us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:26.976482 (+ 1725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:26.977296 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:26.977412 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:26.977525 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0809 09:32:26.978628 (+ 1103us) server_negotiation.cc:299] Negotiation successful
0809 09:32:26.978746 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":460,"thread_start_us":172,"threads_started":1}
I20260809 09:32:26.981592 22008 tablet_service.cc:1467] Processing CreateTablet for tablet 6b8138ea06444b088d95329cac5928d1 (DEFAULT_TABLE table=test-table [id=0e7c5e633694415a806f1267cdd3949f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 09:32:26.981616 22436 tablet_service.cc:1467] Processing CreateTablet for tablet 6b8138ea06444b088d95329cac5928d1 (DEFAULT_TABLE table=test-table [id=0e7c5e633694415a806f1267cdd3949f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 09:32:26.981616 22217 tablet_service.cc:1467] Processing CreateTablet for tablet 6b8138ea06444b088d95329cac5928d1 (DEFAULT_TABLE table=test-table [id=0e7c5e633694415a806f1267cdd3949f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 09:32:26.982723 22008 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6b8138ea06444b088d95329cac5928d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 09:32:26.982733 22436 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6b8138ea06444b088d95329cac5928d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 09:32:26.982728 22217 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6b8138ea06444b088d95329cac5928d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 09:32:26.989260 22558 tablet_bootstrap.cc:492] T 6b8138ea06444b088d95329cac5928d1 P 24aba902e2154a19af6a54cd0f06274f: Bootstrap starting.
I20260809 09:32:26.990844 22559 tablet_bootstrap.cc:492] T 6b8138ea06444b088d95329cac5928d1 P 49c5f666f9e14333b8cbdba292a944a7: Bootstrap starting.
I20260809 09:32:26.991271 22560 tablet_bootstrap.cc:492] T 6b8138ea06444b088d95329cac5928d1 P 4acd715b6e064d6db562024ce1bb0632: Bootstrap starting.
I20260809 09:32:26.992105 22558 tablet_bootstrap.cc:654] T 6b8138ea06444b088d95329cac5928d1 P 24aba902e2154a19af6a54cd0f06274f: Neither blocks nor log segments found. Creating new log.
I20260809 09:32:26.992892 22558 log.cc:824] T 6b8138ea06444b088d95329cac5928d1 P 24aba902e2154a19af6a54cd0f06274f: Log is configured to *not* fsync() on all Append() calls
I20260809 09:32:26.993146 22560 tablet_bootstrap.cc:654] T 6b8138ea06444b088d95329cac5928d1 P 4acd715b6e064d6db562024ce1bb0632: Neither blocks nor log segments found. Creating new log.
I20260809 09:32:26.994514 22560 log.cc:824] T 6b8138ea06444b088d95329cac5928d1 P 4acd715b6e064d6db562024ce1bb0632: Log is configured to *not* fsync() on all Append() calls
I20260809 09:32:26.995388 22558 tablet_bootstrap.cc:492] T 6b8138ea06444b088d95329cac5928d1 P 24aba902e2154a19af6a54cd0f06274f: No bootstrap required, opened a new log
I20260809 09:32:26.995692 22558 ts_tablet_manager.cc:1397] T 6b8138ea06444b088d95329cac5928d1 P 24aba902e2154a19af6a54cd0f06274f: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260809 09:32:26.996505 22559 tablet_bootstrap.cc:654] T 6b8138ea06444b088d95329cac5928d1 P 49c5f666f9e14333b8cbdba292a944a7: Neither blocks nor log segments found. Creating new log.
I20260809 09:32:26.997063 22560 tablet_bootstrap.cc:492] T 6b8138ea06444b088d95329cac5928d1 P 4acd715b6e064d6db562024ce1bb0632: No bootstrap required, opened a new log
I20260809 09:32:26.997282 22560 ts_tablet_manager.cc:1397] T 6b8138ea06444b088d95329cac5928d1 P 4acd715b6e064d6db562024ce1bb0632: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260809 09:32:26.997444 22559 log.cc:824] T 6b8138ea06444b088d95329cac5928d1 P 49c5f666f9e14333b8cbdba292a944a7: Log is configured to *not* fsync() on all Append() calls
I20260809 09:32:26.999452 22559 tablet_bootstrap.cc:492] T 6b8138ea06444b088d95329cac5928d1 P 49c5f666f9e14333b8cbdba292a944a7: No bootstrap required, opened a new log
I20260809 09:32:26.999892 22559 ts_tablet_manager.cc:1397] T 6b8138ea06444b088d95329cac5928d1 P 49c5f666f9e14333b8cbdba292a944a7: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260809 09:32:27.001214 22558 raft_consensus.cc:348] T 6b8138ea06444b088d95329cac5928d1 P 24aba902e2154a19af6a54cd0f06274f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49c5f666f9e14333b8cbdba292a944a7" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 39051 } } peers { permanent_uuid: "24aba902e2154a19af6a54cd0f06274f" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 38863 } } peers { permanent_uuid: "4acd715b6e064d6db562024ce1bb0632" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 38453 } }
I20260809 09:32:27.001654 22558 raft_consensus.cc:374] T 6b8138ea06444b088d95329cac5928d1 P 24aba902e2154a19af6a54cd0f06274f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 09:32:27.001492 22560 raft_consensus.cc:348] T 6b8138ea06444b088d95329cac5928d1 P 4acd715b6e064d6db562024ce1bb0632 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49c5f666f9e14333b8cbdba292a944a7" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 39051 } } peers { permanent_uuid: "24aba902e2154a19af6a54cd0f06274f" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 38863 } } peers { permanent_uuid: "4acd715b6e064d6db562024ce1bb0632" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 38453 } }
I20260809 09:32:27.001756 22560 raft_consensus.cc:374] T 6b8138ea06444b088d95329cac5928d1 P 4acd715b6e064d6db562024ce1bb0632 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 09:32:27.001825 22558 raft_consensus.cc:729] T 6b8138ea06444b088d95329cac5928d1 P 24aba902e2154a19af6a54cd0f06274f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24aba902e2154a19af6a54cd0f06274f, State: Initialized, Role: FOLLOWER
I20260809 09:32:27.001847 22560 raft_consensus.cc:729] T 6b8138ea06444b088d95329cac5928d1 P 4acd715b6e064d6db562024ce1bb0632 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4acd715b6e064d6db562024ce1bb0632, State: Initialized, Role: FOLLOWER
I20260809 09:32:27.002354 22558 consensus_queue.cc:260] T 6b8138ea06444b088d95329cac5928d1 P 24aba902e2154a19af6a54cd0f06274f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49c5f666f9e14333b8cbdba292a944a7" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 39051 } } peers { permanent_uuid: "24aba902e2154a19af6a54cd0f06274f" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 38863 } } peers { permanent_uuid: "4acd715b6e064d6db562024ce1bb0632" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 38453 } }
I20260809 09:32:27.002354 22560 consensus_queue.cc:260] T 6b8138ea06444b088d95329cac5928d1 P 4acd715b6e064d6db562024ce1bb0632 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49c5f666f9e14333b8cbdba292a944a7" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 39051 } } peers { permanent_uuid: "24aba902e2154a19af6a54cd0f06274f" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 38863 } } peers { permanent_uuid: "4acd715b6e064d6db562024ce1bb0632" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 38453 } }
I20260809 09:32:27.003161 22535 heartbeater.cc:503] Master 127.20.125.190:44085 was elected leader, sending a full tablet report...
I20260809 09:32:27.003511 22560 ts_tablet_manager.cc:1428] T 6b8138ea06444b088d95329cac5928d1 P 4acd715b6e064d6db562024ce1bb0632: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260809 09:32:27.003511 22558 ts_tablet_manager.cc:1428] T 6b8138ea06444b088d95329cac5928d1 P 24aba902e2154a19af6a54cd0f06274f: Time spent starting tablet: real 0.008s user 0.007s sys 0.000s
I20260809 09:32:27.003856 22108 heartbeater.cc:503] Master 127.20.125.190:44085 was elected leader, sending a full tablet report...
I20260809 09:32:27.003976 22559 raft_consensus.cc:348] T 6b8138ea06444b088d95329cac5928d1 P 49c5f666f9e14333b8cbdba292a944a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49c5f666f9e14333b8cbdba292a944a7" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 39051 } } peers { permanent_uuid: "24aba902e2154a19af6a54cd0f06274f" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 38863 } } peers { permanent_uuid: "4acd715b6e064d6db562024ce1bb0632" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 38453 } }
I20260809 09:32:27.004262 22559 raft_consensus.cc:374] T 6b8138ea06444b088d95329cac5928d1 P 49c5f666f9e14333b8cbdba292a944a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 09:32:27.004364 22559 raft_consensus.cc:729] T 6b8138ea06444b088d95329cac5928d1 P 49c5f666f9e14333b8cbdba292a944a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49c5f666f9e14333b8cbdba292a944a7, State: Initialized, Role: FOLLOWER
I20260809 09:32:27.005126 22559 consensus_queue.cc:260] T 6b8138ea06444b088d95329cac5928d1 P 49c5f666f9e14333b8cbdba292a944a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49c5f666f9e14333b8cbdba292a944a7" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 39051 } } peers { permanent_uuid: "24aba902e2154a19af6a54cd0f06274f" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 38863 } } peers { permanent_uuid: "4acd715b6e064d6db562024ce1bb0632" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 38453 } }
I20260809 09:32:27.006247 22559 ts_tablet_manager.cc:1428] T 6b8138ea06444b088d95329cac5928d1 P 49c5f666f9e14333b8cbdba292a944a7: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260809 09:32:27.006345 22326 heartbeater.cc:503] Master 127.20.125.190:44085 was elected leader, sending a full tablet report...
I20260809 09:32:27.110919 22571 raft_consensus.cc:482] T 6b8138ea06444b088d95329cac5928d1 P 49c5f666f9e14333b8cbdba292a944a7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 09:32:27.111209 22571 raft_consensus.cc:504] T 6b8138ea06444b088d95329cac5928d1 P 49c5f666f9e14333b8cbdba292a944a7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49c5f666f9e14333b8cbdba292a944a7" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 39051 } } peers { permanent_uuid: "24aba902e2154a19af6a54cd0f06274f" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 38863 } } peers { permanent_uuid: "4acd715b6e064d6db562024ce1bb0632" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 38453 } }
I20260809 09:32:27.112636 22571 leader_election.cc:288] T 6b8138ea06444b088d95329cac5928d1 P 49c5f666f9e14333b8cbdba292a944a7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 24aba902e2154a19af6a54cd0f06274f (127.20.125.129:38863), 4acd715b6e064d6db562024ce1bb0632 (127.20.125.131:38453)
I20260809 09:32:27.116421 22557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:27.113057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.130:53773 (local address 127.20.125.129:38863)
0809 09:32:27.113172 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0809 09:32:27.113177 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 09:32:27.113195 (+ 18us) server_negotiation.cc:407] Connection header received
0809 09:32:27.113327 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:32:27.113330 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:32:27.113374 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:32:27.113474 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 09:32:27.113783 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:27.114311 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:27.115394 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:27.115831 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:27.115968 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:27.116024 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0809 09:32:27.116232 (+ 208us) server_negotiation.cc:299] Negotiation successful
0809 09:32:27.116320 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260809 09:32:27.116415 22330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:27.112935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.129:38863 (local address 127.20.125.130:53773)
0809 09:32:27.113067 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0809 09:32:27.113084 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 09:32:27.113194 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:32:27.113495 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:32:27.113499 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:32:27.113519 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 09:32:27.113646 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:32:27.113652 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:32:27.114434 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:32:27.114439 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:32:27.115244 (+ 805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:32:27.115254 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:32:27.115965 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:32:27.115969 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:32:27.116092 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:27.116111 (+ 19us) client_negotiation.cc:769] Sending connection context
0809 09:32:27.116199 (+ 88us) client_negotiation.cc:240] Negotiation successful
0809 09:32:27.116274 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260809 09:32:27.117230 22572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:27.112938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.131:38453 (local address 127.20.125.130:53757)
0809 09:32:27.113664 (+ 726us) negotiation.cc:107] Waiting for socket to connect
0809 09:32:27.113677 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 09:32:27.113811 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:32:27.114113 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:32:27.114116 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:32:27.114134 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 09:32:27.114277 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:32:27.114283 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:32:27.115138 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:32:27.115143 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:32:27.116010 (+ 867us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:32:27.116019 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:32:27.116881 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:32:27.116884 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:32:27.116995 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:27.117013 (+ 18us) client_negotiation.cc:769] Sending connection context
0809 09:32:27.117081 (+ 68us) client_negotiation.cc:240] Negotiation successful
0809 09:32:27.117126 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":657,"mutex_wait_us":338,"thread_start_us":165,"threads_started":1}
I20260809 09:32:27.117337 22555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:27.113092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.130:53757 (local address 127.20.125.131:38453)
0809 09:32:27.113228 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0809 09:32:27.113233 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 09:32:27.113780 (+ 547us) server_negotiation.cc:407] Connection header received
0809 09:32:27.113943 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:32:27.113949 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:32:27.114005 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:32:27.114099 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 09:32:27.114403 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:27.115011 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:27.116149 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:27.116737 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:27.116874 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:27.116942 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0809 09:32:27.117128 (+ 186us) server_negotiation.cc:299] Negotiation successful
0809 09:32:27.117203 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260809 09:32:27.117293 22037 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6b8138ea06444b088d95329cac5928d1" candidate_uuid: "49c5f666f9e14333b8cbdba292a944a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24aba902e2154a19af6a54cd0f06274f" is_pre_election: true
I20260809 09:32:27.117669 22037 raft_consensus.cc:2454] T 6b8138ea06444b088d95329cac5928d1 P 24aba902e2154a19af6a54cd0f06274f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49c5f666f9e14333b8cbdba292a944a7 in term 0.
I20260809 09:32:27.118019 22468 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6b8138ea06444b088d95329cac5928d1" candidate_uuid: "49c5f666f9e14333b8cbdba292a944a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4acd715b6e064d6db562024ce1bb0632" is_pre_election: true
I20260809 09:32:27.118320 22468 raft_consensus.cc:2454] T 6b8138ea06444b088d95329cac5928d1 P 4acd715b6e064d6db562024ce1bb0632 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49c5f666f9e14333b8cbdba292a944a7 in term 0.
I20260809 09:32:27.118335 22138 leader_election.cc:302] T 6b8138ea06444b088d95329cac5928d1 P 49c5f666f9e14333b8cbdba292a944a7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24aba902e2154a19af6a54cd0f06274f, 49c5f666f9e14333b8cbdba292a944a7; no voters:
I20260809 09:32:27.118664 22571 raft_consensus.cc:2781] T 6b8138ea06444b088d95329cac5928d1 P 49c5f666f9e14333b8cbdba292a944a7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 09:32:27.118777 22571 raft_consensus.cc:482] T 6b8138ea06444b088d95329cac5928d1 P 49c5f666f9e14333b8cbdba292a944a7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 09:32:27.118836 22571 raft_consensus.cc:3037] T 6b8138ea06444b088d95329cac5928d1 P 49c5f666f9e14333b8cbdba292a944a7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 09:32:27.119997 22571 raft_consensus.cc:504] T 6b8138ea06444b088d95329cac5928d1 P 49c5f666f9e14333b8cbdba292a944a7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49c5f666f9e14333b8cbdba292a944a7" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 39051 } } peers { permanent_uuid: "24aba902e2154a19af6a54cd0f06274f" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 38863 } } peers { permanent_uuid: "4acd715b6e064d6db562024ce1bb0632" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 38453 } }
I20260809 09:32:27.120360 22571 leader_election.cc:288] T 6b8138ea06444b088d95329cac5928d1 P 49c5f666f9e14333b8cbdba292a944a7 [CANDIDATE]: Term 1 election: Requested vote from peers 24aba902e2154a19af6a54cd0f06274f (127.20.125.129:38863), 4acd715b6e064d6db562024ce1bb0632 (127.20.125.131:38453)
I20260809 09:32:27.120726 22037 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6b8138ea06444b088d95329cac5928d1" candidate_uuid: "49c5f666f9e14333b8cbdba292a944a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24aba902e2154a19af6a54cd0f06274f"
I20260809 09:32:27.120808 22468 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6b8138ea06444b088d95329cac5928d1" candidate_uuid: "49c5f666f9e14333b8cbdba292a944a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4acd715b6e064d6db562024ce1bb0632"
I20260809 09:32:27.120874 22037 raft_consensus.cc:3037] T 6b8138ea06444b088d95329cac5928d1 P 24aba902e2154a19af6a54cd0f06274f [term 0 FOLLOWER]: Advancing to term 1
I20260809 09:32:27.120949 22468 raft_consensus.cc:3037] T 6b8138ea06444b088d95329cac5928d1 P 4acd715b6e064d6db562024ce1bb0632 [term 0 FOLLOWER]: Advancing to term 1
I20260809 09:32:27.122345 22468 raft_consensus.cc:2454] T 6b8138ea06444b088d95329cac5928d1 P 4acd715b6e064d6db562024ce1bb0632 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49c5f666f9e14333b8cbdba292a944a7 in term 1.
I20260809 09:32:27.122345 22037 raft_consensus.cc:2454] T 6b8138ea06444b088d95329cac5928d1 P 24aba902e2154a19af6a54cd0f06274f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49c5f666f9e14333b8cbdba292a944a7 in term 1.
I20260809 09:32:27.122716 22138 leader_election.cc:302] T 6b8138ea06444b088d95329cac5928d1 P 49c5f666f9e14333b8cbdba292a944a7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24aba902e2154a19af6a54cd0f06274f, 49c5f666f9e14333b8cbdba292a944a7; no voters:
I20260809 09:32:27.122900 22571 raft_consensus.cc:2781] T 6b8138ea06444b088d95329cac5928d1 P 49c5f666f9e14333b8cbdba292a944a7 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 09:32:27.123340 22571 raft_consensus.cc:686] T 6b8138ea06444b088d95329cac5928d1 P 49c5f666f9e14333b8cbdba292a944a7 [term 1 LEADER]: Becoming Leader. State: Replica: 49c5f666f9e14333b8cbdba292a944a7, State: Running, Role: LEADER
I20260809 09:32:27.123739 22571 consensus_queue.cc:237] T 6b8138ea06444b088d95329cac5928d1 P 49c5f666f9e14333b8cbdba292a944a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49c5f666f9e14333b8cbdba292a944a7" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 39051 } } peers { permanent_uuid: "24aba902e2154a19af6a54cd0f06274f" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 38863 } } peers { permanent_uuid: "4acd715b6e064d6db562024ce1bb0632" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 38453 } }
I20260809 09:32:27.128014 21817 catalog_manager.cc:5495] T 6b8138ea06444b088d95329cac5928d1 P 49c5f666f9e14333b8cbdba292a944a7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 49c5f666f9e14333b8cbdba292a944a7 (127.20.125.130). New cstate: current_term: 1 leader_uuid: "49c5f666f9e14333b8cbdba292a944a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49c5f666f9e14333b8cbdba292a944a7" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 39051 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24aba902e2154a19af6a54cd0f06274f" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 38863 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4acd715b6e064d6db562024ce1bb0632" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 38453 } health_report { overall_health: UNKNOWN } } }
W20260809 09:32:27.157812 22537 tablet.cc:2369] T 6b8138ea06444b088d95329cac5928d1 P 4acd715b6e064d6db562024ce1bb0632: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 09 09:32:27 dist-test-slave-7kzw krb5kdc[21757](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786267945, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.125.129@KRBTEST.COM
W20260809 09:32:27.172780 22327 tablet.cc:2369] T 6b8138ea06444b088d95329cac5928d1 P 49c5f666f9e14333b8cbdba292a944a7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 09:32:27.175033 22557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:27.164747 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48968 (local address 127.20.125.129:38863)
0809 09:32:27.164948 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0809 09:32:27.164952 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 09:32:27.165209 (+ 257us) server_negotiation.cc:407] Connection header received
0809 09:32:27.165322 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:32:27.165324 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:32:27.165367 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:32:27.165440 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:32:27.166184 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:27.166882 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:27.167763 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:27.167996 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:27.168082 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:27.171009 (+ 2927us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:32:27.171022 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:32:27.171046 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:32:27.171073 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:32:27.173135 (+ 2062us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:32:27.173614 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:32:27.173621 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:32:27.173623 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:32:27.173727 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:32:27.174018 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:32:27.174021 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:32:27.174023 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:32:27.174289 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:32:27.174530 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0809 09:32:27.174744 (+ 214us) server_negotiation.cc:299] Negotiation successful
0809 09:32:27.174869 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100}
Aug 09 09:32:27 dist-test-slave-7kzw krb5kdc[21757](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786267947, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
W20260809 09:32:27.202601 22109 tablet.cc:2369] T 6b8138ea06444b088d95329cac5928d1 P 24aba902e2154a19af6a54cd0f06274f: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 09 09:32:27 dist-test-slave-7kzw krb5kdc[21757](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786267947, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.20.125.129@KRBTEST.COM
I20260809 09:32:27.209142 22557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:27.199275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48972 (local address 127.20.125.129:38863)
0809 09:32:27.199435 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0809 09:32:27.199440 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 09:32:27.199459 (+ 19us) server_negotiation.cc:407] Connection header received
0809 09:32:27.199630 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:32:27.199632 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:32:27.199679 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:32:27.199759 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:32:27.200519 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:27.201104 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:27.201988 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:27.202241 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:27.202324 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:27.205270 (+ 2946us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:32:27.205274 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:32:27.205276 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:32:27.205304 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:32:27.206960 (+ 1656us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:32:27.207468 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:32:27.207471 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:32:27.207473 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:32:27.207588 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:32:27.208010 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:32:27.208013 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:32:27.208014 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:32:27.208232 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:32:27.208349 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0809 09:32:27.208727 (+ 378us) server_negotiation.cc:299] Negotiation successful
0809 09:32:27.208913 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
Aug 09 09:32:27 dist-test-slave-7kzw krb5kdc[21757](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786267947, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 09:32:27 dist-test-slave-7kzw krb5kdc[21757](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786267947, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.125.129@KRBTEST.COM
I20260809 09:32:27.242952 22557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:27.233602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48980 (local address 127.20.125.129:38863)
0809 09:32:27.233830 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0809 09:32:27.233836 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 09:32:27.233980 (+ 144us) server_negotiation.cc:407] Connection header received
0809 09:32:27.234202 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:32:27.234204 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:32:27.234251 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:32:27.234346 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:32:27.235137 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:27.235638 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:27.236514 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:27.236847 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:27.236993 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:27.239817 (+ 2824us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:32:27.239820 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:32:27.239822 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:32:27.239847 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:32:27.241189 (+ 1342us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:32:27.241630 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:32:27.241634 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:32:27.241636 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:32:27.241726 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:32:27.242013 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:32:27.242018 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:32:27.242019 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:32:27.242196 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:32:27.242335 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0809 09:32:27.242617 (+ 282us) server_negotiation.cc:299] Negotiation successful
0809 09:32:27.242791 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111}
W20260809 09:32:27.244632 21978 tablet_service.cc:2908] Rejecting scan request for tablet 6b8138ea06444b088d95329cac5928d1: Uninitialized: clean time has not yet been initialized
I20260809 09:32:27.247069 20982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu with pid 21915
I20260809 09:32:27.271809 20982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu with pid 22116
I20260809 09:32:27.285535 20982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu with pid 22333
I20260809 09:32:27.299458 20982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu with pid 21773
2026-08-09T09:32:27Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2769 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786267941944493-20982-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 09:32:27 dist-test-slave-7kzw krb5kdc[22595](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 09:32:27 dist-test-slave-7kzw krb5kdc[22595](info): set up 2 sockets
Aug 09 09:32:27 dist-test-slave-7kzw krb5kdc[22595](info): commencing operation
krb5kdc: starting...
W20260809 09:32:28.553634 20982 mini_kdc.cc:121] Time spent starting KDC: real 1.226s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786267941944493-20982-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786267941944493-20982-0/krb5kdc/test-user.keytab.
Aug 09 09:32:28 dist-test-slave-7kzw krb5kdc[22595](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786267948, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T09:32:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T09:32:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.125.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.125.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.125.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.190.keytab.
Entry for principal kudu/127.20.125.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.190.keytab.
I20260809 09:32:28.694152 20982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
/tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786267941944493-20982-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786267941944493-20982-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786267941944493-20982-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786267941944493-20982-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.125.190:44501
--webserver_interface=127.20.125.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.190.keytab
--principal=kudu/127.20.125.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.125.148:37095
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.125.190:44501
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786267941944493-20982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786267941944493-20982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786267941944493-20982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 09:32:28.813024 22617 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 09:32:28.813373 22617 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 09:32:28.813473 22617 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 09:32:28.817605 22617 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 09:32:28.817713 22617 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260809 09:32:28.817801 22617 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 09:32:28.817847 22617 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 09:32:28.817904 22617 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 09:32:28.817941 22617 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 09:32:28.823444 22617 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.125.148:37095
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786267941944493-20982-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786267941944493-20982-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.125.190:44501
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.190.keytab
--principal=kudu/127.20.125.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.125.190:44501
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786267941944493-20982-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.125.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22617
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786267941944493-20982-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 08:38:55 UTC on ccycloud.quasar-njjleu.root.comops.site
build id 4575935
I20260809 09:32:28.824584 22617 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 09:32:28.825659 22617 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 09:32:28.834666 22625 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:32:28.834775 22617 server_base.cc:1034] running on GCE node
W20260809 09:32:28.834705 22626 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:32:28.834666 22629 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:32:28.835604 22617 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 09:32:28.836764 22617 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 09:32:28.837978 22617 hybrid_clock.cc:648] HybridClock initialized: now 1786267948837957 us; error 38 us; skew 500 ppm
Aug 09 09:32:28 dist-test-slave-7kzw krb5kdc[22595](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786267948, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 09:32:28.845402 22617 init.cc:377] Logged in from keytab as kudu/127.20.125.190@KRBTEST.COM (short username kudu)
I20260809 09:32:28.846524 22617 webserver.cc:460] Webserver started at http://127.20.125.190:45289/ using document root <none> and password file <none>
I20260809 09:32:28.847144 22617 fs_manager.cc:359] Metadata directory not provided
I20260809 09:32:28.847332 22617 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 09:32:28.847644 22617 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 09:32:28.850206 22617 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786267941944493-20982-0/minicluster-data/master-0/data/instance:
uuid: "0ef673811e7443ac84e6e78369057d13"
format_stamp: "Formatted at 2026-08-09 09:32:28 on dist-test-slave-7kzw"
server_key: "b2acb0893edaad24036b4c5d807526f5"
server_key_iv: "dae4ee377b366024ed4b18a9e07cdd92"
server_key_version: "encryptionkey@0"
I20260809 09:32:28.851050 22617 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786267941944493-20982-0/minicluster-data/master-0/wal/instance:
uuid: "0ef673811e7443ac84e6e78369057d13"
format_stamp: "Formatted at 2026-08-09 09:32:28 on dist-test-slave-7kzw"
server_key: "b2acb0893edaad24036b4c5d807526f5"
server_key_iv: "dae4ee377b366024ed4b18a9e07cdd92"
server_key_version: "encryptionkey@0"
I20260809 09:32:28.855413 22617 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260809 09:32:28.858228 22637 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 09:32:28.859772 22617 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 09:32:28.859946 22617 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786267941944493-20982-0/minicluster-data/master-0/data,/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786267941944493-20982-0/minicluster-data/master-0/wal
uuid: "0ef673811e7443ac84e6e78369057d13"
format_stamp: "Formatted at 2026-08-09 09:32:28 on dist-test-slave-7kzw"
server_key: "b2acb0893edaad24036b4c5d807526f5"
server_key_iv: "dae4ee377b366024ed4b18a9e07cdd92"
server_key_version: "encryptionkey@0"
I20260809 09:32:28.860077 22617 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786267941944493-20982-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786267941944493-20982-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786267941944493-20982-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 09:32:28.879311 22617 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 09:32:28.883162 22617 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 09:32:28.883404 22617 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 09:32:28.892134 22617 rpc_server.cc:307] RPC server started. Bound to: 127.20.125.190:44501
I20260809 09:32:28.892195 22731 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.125.190:44501 every 8 connection(s)
I20260809 09:32:28.893213 22617 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786267941944493-20982-0/minicluster-data/master-0/data/info.pb
I20260809 09:32:28.896368 22733 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 09:32:28.902765 22733 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0ef673811e7443ac84e6e78369057d13: Bootstrap starting.
I20260809 09:32:28.903261 20982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu as pid 22617
I20260809 09:32:28.903401 20982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786267941944493-20982-0/minicluster-data/master-0/wal/instance
I20260809 09:32:28.903687 20982 external_mini_cluster.cc:1428] Setting key 98869aa314f0870e29416677aa5f0cdf
I20260809 09:32:28.906014 22733 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0ef673811e7443ac84e6e78369057d13: Neither blocks nor log segments found. Creating new log.
I20260809 09:32:28.907003 22733 log.cc:824] T 00000000000000000000000000000000 P 0ef673811e7443ac84e6e78369057d13: Log is configured to *not* fsync() on all Append() calls
I20260809 09:32:28.909253 22733 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0ef673811e7443ac84e6e78369057d13: No bootstrap required, opened a new log
Aug 09 09:32:28 dist-test-slave-7kzw krb5kdc[22595](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786267948, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.125.190@KRBTEST.COM
I20260809 09:32:28.913668 22733 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0ef673811e7443ac84e6e78369057d13 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ef673811e7443ac84e6e78369057d13" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 44501 } }
I20260809 09:32:28.914054 22733 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0ef673811e7443ac84e6e78369057d13 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 09:32:28.914151 22733 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0ef673811e7443ac84e6e78369057d13 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ef673811e7443ac84e6e78369057d13, State: Initialized, Role: FOLLOWER
I20260809 09:32:28.914706 22733 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0ef673811e7443ac84e6e78369057d13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ef673811e7443ac84e6e78369057d13" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 44501 } }
I20260809 09:32:28.914896 22733 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0ef673811e7443ac84e6e78369057d13 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 09:32:28.914999 22733 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0ef673811e7443ac84e6e78369057d13 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 09:32:28.915136 22733 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0ef673811e7443ac84e6e78369057d13 [term 0 FOLLOWER]: Advancing to term 1
I20260809 09:32:28.916190 22733 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0ef673811e7443ac84e6e78369057d13 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ef673811e7443ac84e6e78369057d13" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 44501 } }
I20260809 09:32:28.916579 22733 leader_election.cc:302] T 00000000000000000000000000000000 P 0ef673811e7443ac84e6e78369057d13 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0ef673811e7443ac84e6e78369057d13; no voters:
I20260809 09:32:28.916991 22733 leader_election.cc:288] T 00000000000000000000000000000000 P 0ef673811e7443ac84e6e78369057d13 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 09:32:28.917088 22744 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0ef673811e7443ac84e6e78369057d13 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 09:32:28.917326 22744 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0ef673811e7443ac84e6e78369057d13 [term 1 LEADER]: Becoming Leader. State: Replica: 0ef673811e7443ac84e6e78369057d13, State: Running, Role: LEADER
I20260809 09:32:28.917728 22744 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0ef673811e7443ac84e6e78369057d13 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ef673811e7443ac84e6e78369057d13" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 44501 } }
I20260809 09:32:28.918557 22733 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0ef673811e7443ac84e6e78369057d13 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 09:32:28.919853 22745 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0ef673811e7443ac84e6e78369057d13 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0ef673811e7443ac84e6e78369057d13. Latest consensus state: current_term: 1 leader_uuid: "0ef673811e7443ac84e6e78369057d13" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ef673811e7443ac84e6e78369057d13" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 44501 } } }
I20260809 09:32:28.919875 22747 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0ef673811e7443ac84e6e78369057d13 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0ef673811e7443ac84e6e78369057d13" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ef673811e7443ac84e6e78369057d13" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 44501 } } }
I20260809 09:32:28.920068 22745 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0ef673811e7443ac84e6e78369057d13 [sys.catalog]: This master's current role is: LEADER
I20260809 09:32:28.920097 22747 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0ef673811e7443ac84e6e78369057d13 [sys.catalog]: This master's current role is: LEADER
I20260809 09:32:28.920526 22758 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 09:32:28.920650 22740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:28.905299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55590 (local address 127.20.125.190:44501)
0809 09:32:28.906012 (+ 713us) server_negotiation.cc:206] Beginning negotiation
0809 09:32:28.906033 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0809 09:32:28.906075 (+ 42us) server_negotiation.cc:407] Connection header received
0809 09:32:28.907025 (+ 950us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:32:28.907049 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:32:28.907499 (+ 450us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:32:28.907880 (+ 381us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:32:28.908751 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:28.909919 (+ 1168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:28.910924 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:28.911423 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:28.911618 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:28.914816 (+ 3198us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:32:28.914822 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:32:28.914836 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:32:28.914865 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:32:28.916968 (+ 2103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:32:28.917602 (+ 634us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:32:28.917611 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:32:28.917618 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:32:28.917774 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:32:28.918097 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:32:28.918100 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:32:28.918102 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:32:28.918638 (+ 536us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:32:28.918840 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0809 09:32:28.919158 (+ 318us) server_negotiation.cc:299] Negotiation successful
0809 09:32:28.919496 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":380,"thread_start_us":159,"threads_started":1}
I20260809 09:32:28.924098 22758 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 09:32:28.929970 22758 catalog_manager.cc:1347] Generated new cluster ID: 3d8e71ab3b5e4afd98eb08a8cb867e0f
I20260809 09:32:28.930051 22758 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 09:32:28.941753 22758 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 09:32:28.942926 22758 catalog_manager.cc:1504] Loading token signing keys...
I20260809 09:32:28.949429 22758 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0ef673811e7443ac84e6e78369057d13: Generated new TSK 0
I20260809 09:32:28.950136 22758 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.20.125.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.125.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.125.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.129.keytab.
Entry for principal kudu/127.20.125.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.129.keytab.
I20260809 09:32:28.987501 20982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
/tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786267941944493-20982-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786267941944493-20982-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786267941944493-20982-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786267941944493-20982-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.125.129:0
--local_ip_for_outbound_sockets=127.20.125.129
--webserver_interface=127.20.125.129
--webserver_port=0
--tserver_master_addrs=127.20.125.190:44501
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.129.keytab
--principal=kudu/127.20.125.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.125.148:37095
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786267941944493-20982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786267941944493-20982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786267941944493-20982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 09:32:29.132870 22772 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 09:32:29.133304 22772 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 09:32:29.133406 22772 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 09:32:29.138302 22772 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 09:32:29.138402 22772 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 09:32:29.138475 22772 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 09:32:29.138562 22772 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.125.129
I20260809 09:32:29.143903 22772 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.125.148:37095
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786267941944493-20982-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786267941944493-20982-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.129.keytab
--principal=kudu/127.20.125.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.125.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786267941944493-20982-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.125.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.20.125.190:44501
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22772
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.125.129
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786267941944493-20982-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 08:38:55 UTC on ccycloud.quasar-njjleu.root.comops.site
build id 4575935
I20260809 09:32:29.144963 22772 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 09:32:29.145998 22772 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 09:32:29.156002 22772 server_base.cc:1034] running on GCE node
W20260809 09:32:29.155962 22777 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:32:29.155962 22778 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:32:29.155984 22782 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:32:29.156703 22772 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 09:32:29.157356 22772 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 09:32:29.158557 22772 hybrid_clock.cc:648] HybridClock initialized: now 1786267949158528 us; error 40 us; skew 500 ppm
Aug 09 09:32:29 dist-test-slave-7kzw krb5kdc[22595](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786267949, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 09:32:29.166144 22772 init.cc:377] Logged in from keytab as kudu/127.20.125.129@KRBTEST.COM (short username kudu)
I20260809 09:32:29.167088 22772 webserver.cc:460] Webserver started at http://127.20.125.129:42643/ using document root <none> and password file <none>
I20260809 09:32:29.167686 22772 fs_manager.cc:359] Metadata directory not provided
I20260809 09:32:29.167781 22772 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 09:32:29.168033 22772 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 09:32:29.170082 22772 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786267941944493-20982-0/minicluster-data/ts-0/data/instance:
uuid: "c405fafe9cca4eeea6ff3e56a6f8f61a"
format_stamp: "Formatted at 2026-08-09 09:32:29 on dist-test-slave-7kzw"
server_key: "9af8c88f006e98d7c0af6d2d970a899c"
server_key_iv: "751ff586d071cc0c65b15efdd696e246"
server_key_version: "encryptionkey@0"
I20260809 09:32:29.170717 22772 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786267941944493-20982-0/minicluster-data/ts-0/wal/instance:
uuid: "c405fafe9cca4eeea6ff3e56a6f8f61a"
format_stamp: "Formatted at 2026-08-09 09:32:29 on dist-test-slave-7kzw"
server_key: "9af8c88f006e98d7c0af6d2d970a899c"
server_key_iv: "751ff586d071cc0c65b15efdd696e246"
server_key_version: "encryptionkey@0"
I20260809 09:32:29.174528 22772 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260809 09:32:29.177018 22793 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 09:32:29.178285 22772 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 09:32:29.178429 22772 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786267941944493-20982-0/minicluster-data/ts-0/data,/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786267941944493-20982-0/minicluster-data/ts-0/wal
uuid: "c405fafe9cca4eeea6ff3e56a6f8f61a"
format_stamp: "Formatted at 2026-08-09 09:32:29 on dist-test-slave-7kzw"
server_key: "9af8c88f006e98d7c0af6d2d970a899c"
server_key_iv: "751ff586d071cc0c65b15efdd696e246"
server_key_version: "encryptionkey@0"
I20260809 09:32:29.178594 22772 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786267941944493-20982-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786267941944493-20982-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786267941944493-20982-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 09:32:29.192502 22772 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 09:32:29.196014 22772 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 09:32:29.196262 22772 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 09:32:29.197847 22772 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 09:32:29.197930 22772 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 09:32:29.198002 22772 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 09:32:29.198050 22772 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 09:32:29 dist-test-slave-7kzw krb5kdc[22595](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786267949, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.129@KRBTEST.COM for kudu/127.20.125.190@KRBTEST.COM
I20260809 09:32:29.210426 22985 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.125.129:36447 every 8 connection(s)
I20260809 09:32:29.210423 22772 rpc_server.cc:307] RPC server started. Bound to: 127.20.125.129:36447
I20260809 09:32:29.211763 22772 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786267941944493-20982-0/minicluster-data/ts-0/data/info.pb
I20260809 09:32:29.213132 22740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:29.199010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.129:45709 (local address 127.20.125.190:44501)
0809 09:32:29.199140 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0809 09:32:29.199143 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 09:32:29.200289 (+ 1146us) server_negotiation.cc:407] Connection header received
0809 09:32:29.201379 (+ 1090us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:32:29.201385 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:32:29.201453 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:32:29.201514 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:32:29.203378 (+ 1864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:29.203969 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:29.205128 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:29.205423 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:29.205541 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:29.208934 (+ 3393us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:32:29.208938 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:32:29.208940 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:32:29.208976 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:32:29.210864 (+ 1888us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:32:29.211378 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:32:29.211383 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:32:29.211385 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:32:29.211488 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:32:29.211867 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:32:29.211871 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:32:29.211873 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:32:29.212146 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:32:29.212268 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0809 09:32:29.212862 (+ 594us) server_negotiation.cc:299] Negotiation successful
0809 09:32:29.212989 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260809 09:32:29.213914 22814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:29.199599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:44501 (local address 127.20.125.129:45709)
0809 09:32:29.200119 (+ 520us) negotiation.cc:107] Waiting for socket to connect
0809 09:32:29.200162 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 09:32:29.201083 (+ 921us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:32:29.201982 (+ 899us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:32:29.201991 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:32:29.202524 (+ 533us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:32:29.203194 (+ 670us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:32:29.203216 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:32:29.204104 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:32:29.204107 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:32:29.204991 (+ 884us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:32:29.205001 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:32:29.205602 (+ 601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:32:29.205606 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:32:29.205780 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:29.206513 (+ 733us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 09:32:29.206533 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:32:29.208757 (+ 2224us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 09:32:29.210986 (+ 2229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:32:29.210991 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:32:29.211002 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:32:29.211262 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:32:29.211589 (+ 327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:32:29.211591 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:32:29.211592 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:32:29.211757 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:32:29.212284 (+ 527us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 09:32:29.212290 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 09:32:29.212562 (+ 272us) client_negotiation.cc:769] Sending connection context
0809 09:32:29.212803 (+ 241us) client_negotiation.cc:240] Negotiation successful
0809 09:32:29.213067 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":322,"thread_start_us":150,"threads_started":1}
I20260809 09:32:29.215453 22986 heartbeater.cc:347] Connected to a master server at 127.20.125.190:44501
I20260809 09:32:29.215637 22986 heartbeater.cc:464] Registering TS with master...
I20260809 09:32:29.216172 22986 heartbeater.cc:511] Master 127.20.125.190:44501 requested a full tablet report, sending...
I20260809 09:32:29.216956 20982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu as pid 22772
I20260809 09:32:29.217084 20982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786267941944493-20982-0/minicluster-data/ts-0/wal/instance
I20260809 09:32:29.217401 20982 external_mini_cluster.cc:1428] Setting key b0d2e2a52a44b2fdea854707bd20a3b6
I20260809 09:32:29.217908 22664 ts_manager.cc:194] Registered new tserver with Master: c405fafe9cca4eeea6ff3e56a6f8f61a (127.20.125.129:36447)
I20260809 09:32:29.219482 22664 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.125.129@KRBTEST.COM'} at 127.20.125.129:45709
I20260809 09:32:29.227445 22740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:29.218209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.129:35899 (local address 127.20.125.190:44501)
0809 09:32:29.218321 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0809 09:32:29.218325 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 09:32:29.218959 (+ 634us) server_negotiation.cc:407] Connection header received
0809 09:32:29.219090 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:32:29.219094 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:32:29.219144 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:32:29.219238 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:32:29.220036 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:29.220883 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:29.222057 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:29.222363 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:29.222452 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:29.223973 (+ 1521us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:32:29.223980 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:32:29.223985 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:32:29.224018 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:32:29.225667 (+ 1649us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:32:29.226129 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:32:29.226138 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:32:29.226139 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:32:29.226224 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:32:29.226566 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:32:29.226570 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:32:29.226571 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:32:29.226762 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:32:29.226876 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0809 09:32:29.227151 (+ 275us) server_negotiation.cc:299] Negotiation successful
0809 09:32:29.227285 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260809 09:32:29.227455 22995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:29.218078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:44501 (local address 127.20.125.129:35899)
0809 09:32:29.218859 (+ 781us) negotiation.cc:107] Waiting for socket to connect
0809 09:32:29.218876 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 09:32:29.218981 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:32:29.219316 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:32:29.219319 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:32:29.219547 (+ 228us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:32:29.219846 (+ 299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:32:29.219862 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:32:29.221050 (+ 1188us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:32:29.221054 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:32:29.221908 (+ 854us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:32:29.221924 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:32:29.222518 (+ 594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:32:29.222524 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:32:29.222638 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:29.223427 (+ 789us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 09:32:29.223439 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:32:29.223816 (+ 377us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 09:32:29.225799 (+ 1983us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:32:29.225803 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:32:29.225805 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:32:29.226017 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:32:29.226321 (+ 304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:32:29.226324 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:32:29.226326 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:32:29.226420 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:32:29.226867 (+ 447us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 09:32:29.226870 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 09:32:29.226974 (+ 104us) client_negotiation.cc:769] Sending connection context
0809 09:32:29.227083 (+ 109us) client_negotiation.cc:240] Negotiation successful
0809 09:32:29.227234 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":663,"mutex_wait_us":3,"thread_start_us":106,"threads_started":1}
WARNING: no policy specified for kudu/127.20.125.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.125.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.125.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.130.keytab.
Entry for principal kudu/127.20.125.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.130.keytab.
I20260809 09:32:29.251494 20982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
/tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786267941944493-20982-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786267941944493-20982-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786267941944493-20982-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786267941944493-20982-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.125.130:0
--local_ip_for_outbound_sockets=127.20.125.130
--webserver_interface=127.20.125.130
--webserver_port=0
--tserver_master_addrs=127.20.125.190:44501
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.130.keytab
--principal=kudu/127.20.125.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.125.148:37095
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786267941944493-20982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786267941944493-20982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786267941944493-20982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 09:32:29.370520 23001 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 09:32:29.370800 23001 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 09:32:29.370846 23001 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 09:32:29.374786 23001 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 09:32:29.374847 23001 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 09:32:29.374886 23001 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 09:32:29.374922 23001 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.125.130
I20260809 09:32:29.380154 23001 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.125.148:37095
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786267941944493-20982-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786267941944493-20982-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.130.keytab
--principal=kudu/127.20.125.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.125.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786267941944493-20982-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.125.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.20.125.190:44501
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23001
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.125.130
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786267941944493-20982-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 08:38:55 UTC on ccycloud.quasar-njjleu.root.comops.site
build id 4575935
I20260809 09:32:29.381174 23001 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 09:32:29.382225 23001 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 09:32:29.392357 23019 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:32:29.392357 23012 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:32:29.392364 23013 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:32:29.392697 23001 server_base.cc:1034] running on GCE node
I20260809 09:32:29.393215 23001 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 09:32:29.393942 23001 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 09:32:29.395117 23001 hybrid_clock.cc:648] HybridClock initialized: now 1786267949395075 us; error 51 us; skew 500 ppm
Aug 09 09:32:29 dist-test-slave-7kzw krb5kdc[22595](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786267949, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 09:32:29.398228 23001 init.cc:377] Logged in from keytab as kudu/127.20.125.130@KRBTEST.COM (short username kudu)
I20260809 09:32:29.399240 23001 webserver.cc:460] Webserver started at http://127.20.125.130:45125/ using document root <none> and password file <none>
I20260809 09:32:29.399814 23001 fs_manager.cc:359] Metadata directory not provided
I20260809 09:32:29.399912 23001 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 09:32:29.400159 23001 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 09:32:29.402212 23001 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786267941944493-20982-0/minicluster-data/ts-1/data/instance:
uuid: "41c5693686644259835b5ee95bca2fad"
format_stamp: "Formatted at 2026-08-09 09:32:29 on dist-test-slave-7kzw"
server_key: "66f711c78eb517482bc0444cb43dd418"
server_key_iv: "8e0fcadb08d9b40bd87849da3816c421"
server_key_version: "encryptionkey@0"
I20260809 09:32:29.402832 23001 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786267941944493-20982-0/minicluster-data/ts-1/wal/instance:
uuid: "41c5693686644259835b5ee95bca2fad"
format_stamp: "Formatted at 2026-08-09 09:32:29 on dist-test-slave-7kzw"
server_key: "66f711c78eb517482bc0444cb43dd418"
server_key_iv: "8e0fcadb08d9b40bd87849da3816c421"
server_key_version: "encryptionkey@0"
I20260809 09:32:29.406793 23001 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260809 09:32:29.409386 23033 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 09:32:29.410707 23001 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 09:32:29.410848 23001 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786267941944493-20982-0/minicluster-data/ts-1/data,/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786267941944493-20982-0/minicluster-data/ts-1/wal
uuid: "41c5693686644259835b5ee95bca2fad"
format_stamp: "Formatted at 2026-08-09 09:32:29 on dist-test-slave-7kzw"
server_key: "66f711c78eb517482bc0444cb43dd418"
server_key_iv: "8e0fcadb08d9b40bd87849da3816c421"
server_key_version: "encryptionkey@0"
I20260809 09:32:29.410969 23001 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786267941944493-20982-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786267941944493-20982-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786267941944493-20982-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 09:32:29.424515 23001 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 09:32:29.428079 23001 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 09:32:29.428288 23001 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 09:32:29.429749 23001 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 09:32:29.429802 23001 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 09:32:29.429870 23001 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 09:32:29.429904 23001 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 09:32:29.441220 23001 rpc_server.cc:307] RPC server started. Bound to: 127.20.125.130:46859
I20260809 09:32:29.441300 23210 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.125.130:46859 every 8 connection(s)
Aug 09 09:32:29 dist-test-slave-7kzw krb5kdc[22595](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786267949, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.130@KRBTEST.COM for kudu/127.20.125.190@KRBTEST.COM
I20260809 09:32:29.442348 23001 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786267941944493-20982-0/minicluster-data/ts-1/data/info.pb
I20260809 09:32:29.445907 22740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:29.430838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.130:52093 (local address 127.20.125.190:44501)
0809 09:32:29.430990 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0809 09:32:29.430995 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 09:32:29.432011 (+ 1016us) server_negotiation.cc:407] Connection header received
0809 09:32:29.433508 (+ 1497us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:32:29.433514 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:32:29.433581 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:32:29.433704 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:32:29.435461 (+ 1757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:29.436373 (+ 912us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:29.438312 (+ 1939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:29.438643 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:29.438782 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:29.442196 (+ 3414us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:32:29.442200 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:32:29.442202 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:32:29.442229 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:32:29.443955 (+ 1726us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:32:29.444422 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:32:29.444425 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:32:29.444426 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:32:29.444506 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:32:29.444840 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:32:29.444842 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:32:29.444843 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:32:29.445024 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:32:29.445099 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0809 09:32:29.445644 (+ 545us) server_negotiation.cc:299] Negotiation successful
0809 09:32:29.445776 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 09:32:29.446746 23053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:29.431275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:44501 (local address 127.20.125.130:52093)
0809 09:32:29.431846 (+ 571us) negotiation.cc:107] Waiting for socket to connect
0809 09:32:29.431902 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0809 09:32:29.433204 (+ 1302us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:32:29.433907 (+ 703us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:32:29.433925 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:32:29.434495 (+ 570us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:32:29.435270 (+ 775us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:32:29.435294 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:32:29.436526 (+ 1232us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:32:29.436529 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:32:29.438153 (+ 1624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:32:29.438166 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:32:29.438839 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:32:29.438843 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:32:29.439099 (+ 256us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:29.439921 (+ 822us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 09:32:29.439950 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:32:29.441953 (+ 2003us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 09:32:29.444057 (+ 2104us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:32:29.444061 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:32:29.444072 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:32:29.444331 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:32:29.444585 (+ 254us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:32:29.444587 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:32:29.444588 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:32:29.444755 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:32:29.445112 (+ 357us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 09:32:29.445117 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 09:32:29.445345 (+ 228us) client_negotiation.cc:769] Sending connection context
0809 09:32:29.445502 (+ 157us) client_negotiation.cc:240] Negotiation successful
0809 09:32:29.445784 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":331,"thread_start_us":135,"threads_started":1}
I20260809 09:32:29.448060 23212 heartbeater.cc:347] Connected to a master server at 127.20.125.190:44501
I20260809 09:32:29.448233 23212 heartbeater.cc:464] Registering TS with master...
I20260809 09:32:29.448840 23212 heartbeater.cc:511] Master 127.20.125.190:44501 requested a full tablet report, sending...
I20260809 09:32:29.449537 20982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu as pid 23001
I20260809 09:32:29.449642 20982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786267941944493-20982-0/minicluster-data/ts-1/wal/instance
I20260809 09:32:29.449937 20982 external_mini_cluster.cc:1428] Setting key 4cdd3beda49f3d6201ea6e669e17fe32
I20260809 09:32:29.450057 22666 ts_manager.cc:194] Registered new tserver with Master: 41c5693686644259835b5ee95bca2fad (127.20.125.130:46859)
I20260809 09:32:29.450830 22666 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.125.130@KRBTEST.COM'} at 127.20.125.130:52093
I20260809 09:32:29.458323 23223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:29.450610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:44501 (local address 127.20.125.130:57261)
0809 09:32:29.451026 (+ 416us) negotiation.cc:107] Waiting for socket to connect
0809 09:32:29.451044 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 09:32:29.451163 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:32:29.451423 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:32:29.451426 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:32:29.451632 (+ 206us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:32:29.451732 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:32:29.451738 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:32:29.452482 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:32:29.452485 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:32:29.453032 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:32:29.453040 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:32:29.453511 (+ 471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:32:29.453514 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:32:29.453616 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:29.454152 (+ 536us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 09:32:29.454163 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:32:29.454543 (+ 380us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 09:32:29.456578 (+ 2035us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:32:29.456581 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:32:29.456589 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:32:29.456836 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:32:29.457162 (+ 326us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:32:29.457164 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:32:29.457166 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:32:29.457249 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:32:29.457834 (+ 585us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 09:32:29.457836 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 09:32:29.457928 (+ 92us) client_negotiation.cc:769] Sending connection context
0809 09:32:29.458052 (+ 124us) client_negotiation.cc:240] Negotiation successful
0809 09:32:29.458178 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":244,"thread_start_us":116,"threads_started":1}
I20260809 09:32:29.458546 22740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:29.450663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.130:57261 (local address 127.20.125.190:44501)
0809 09:32:29.450784 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0809 09:32:29.450789 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 09:32:29.451120 (+ 331us) server_negotiation.cc:407] Connection header received
0809 09:32:29.451275 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:32:29.451278 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:32:29.451319 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:32:29.451406 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:32:29.451850 (+ 444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:29.452354 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:29.453162 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:29.453391 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:29.453495 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:29.454684 (+ 1189us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:32:29.454688 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:32:29.454691 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:32:29.454715 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:32:29.456437 (+ 1722us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:32:29.456952 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:32:29.456958 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:32:29.456960 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:32:29.457062 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:32:29.457352 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:32:29.457357 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:32:29.457359 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:32:29.457694 (+ 335us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:32:29.457812 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0809 09:32:29.458144 (+ 332us) server_negotiation.cc:299] Negotiation successful
0809 09:32:29.458284 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
WARNING: no policy specified for kudu/127.20.125.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.125.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.125.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.131.keytab.
Entry for principal kudu/127.20.125.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.131.keytab.
I20260809 09:32:29.481782 20982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
/tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786267941944493-20982-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786267941944493-20982-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786267941944493-20982-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786267941944493-20982-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.125.131:0
--local_ip_for_outbound_sockets=127.20.125.131
--webserver_interface=127.20.125.131
--webserver_port=0
--tserver_master_addrs=127.20.125.190:44501
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.131.keytab
--principal=kudu/127.20.125.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.125.148:37095
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786267941944493-20982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786267941944493-20982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786267941944493-20982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 09:32:29.598193 23227 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 09:32:29.598438 23227 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 09:32:29.598555 23227 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 09:32:29.602757 23227 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 09:32:29.602818 23227 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 09:32:29.602857 23227 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 09:32:29.602890 23227 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.125.131
I20260809 09:32:29.608040 23227 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.125.148:37095
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786267941944493-20982-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786267941944493-20982-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.131.keytab
--principal=kudu/127.20.125.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.125.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786267941944493-20982-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.125.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.20.125.190:44501
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23227
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.125.131
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786267941944493-20982-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 08:38:55 UTC on ccycloud.quasar-njjleu.root.comops.site
build id 4575935
I20260809 09:32:29.609112 23227 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 09:32:29.610116 23227 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 09:32:29.620450 23240 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:32:29.620574 23242 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:32:29.620458 23236 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:32:29.620801 23227 server_base.cc:1034] running on GCE node
I20260809 09:32:29.621236 23227 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 09:32:29.621889 23227 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 09:32:29.623131 23227 hybrid_clock.cc:648] HybridClock initialized: now 1786267949623123 us; error 36 us; skew 500 ppm
Aug 09 09:32:29 dist-test-slave-7kzw krb5kdc[22595](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786267949, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 09:32:29.626039 23227 init.cc:377] Logged in from keytab as kudu/127.20.125.131@KRBTEST.COM (short username kudu)
I20260809 09:32:29.627070 23227 webserver.cc:460] Webserver started at http://127.20.125.131:41765/ using document root <none> and password file <none>
I20260809 09:32:29.627628 23227 fs_manager.cc:359] Metadata directory not provided
I20260809 09:32:29.627692 23227 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 09:32:29.627908 23227 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 09:32:29.629874 23227 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786267941944493-20982-0/minicluster-data/ts-2/data/instance:
uuid: "f326fb02673b498d958b36e7fd8eb682"
format_stamp: "Formatted at 2026-08-09 09:32:29 on dist-test-slave-7kzw"
server_key: "a82d34cfe31a290eae068f6f7c93be3b"
server_key_iv: "7fb142e7548d8adee67ea23dd97eb4e1"
server_key_version: "encryptionkey@0"
I20260809 09:32:29.630456 23227 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786267941944493-20982-0/minicluster-data/ts-2/wal/instance:
uuid: "f326fb02673b498d958b36e7fd8eb682"
format_stamp: "Formatted at 2026-08-09 09:32:29 on dist-test-slave-7kzw"
server_key: "a82d34cfe31a290eae068f6f7c93be3b"
server_key_iv: "7fb142e7548d8adee67ea23dd97eb4e1"
server_key_version: "encryptionkey@0"
I20260809 09:32:29.634372 23227 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260809 09:32:29.637100 23255 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 09:32:29.638800 23227 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 09:32:29.638921 23227 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786267941944493-20982-0/minicluster-data/ts-2/data,/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786267941944493-20982-0/minicluster-data/ts-2/wal
uuid: "f326fb02673b498d958b36e7fd8eb682"
format_stamp: "Formatted at 2026-08-09 09:32:29 on dist-test-slave-7kzw"
server_key: "a82d34cfe31a290eae068f6f7c93be3b"
server_key_iv: "7fb142e7548d8adee67ea23dd97eb4e1"
server_key_version: "encryptionkey@0"
I20260809 09:32:29.639065 23227 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786267941944493-20982-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786267941944493-20982-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786267941944493-20982-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 09:32:29.655493 23227 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 09:32:29.659055 23227 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 09:32:29.659376 23227 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 09:32:29.660946 23227 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 09:32:29.661015 23227 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 09:32:29.661072 23227 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 09:32:29.661101 23227 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 09:32:29 dist-test-slave-7kzw krb5kdc[22595](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786267949, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.131@KRBTEST.COM for kudu/127.20.125.190@KRBTEST.COM
I20260809 09:32:29.673827 23227 rpc_server.cc:307] RPC server started. Bound to: 127.20.125.131:42809
I20260809 09:32:29.673964 23431 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.125.131:42809 every 8 connection(s)
I20260809 09:32:29.674870 23227 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786267941944493-20982-0/minicluster-data/ts-2/data/info.pb
I20260809 09:32:29.678107 22740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:29.661899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.131:37097 (local address 127.20.125.190:44501)
0809 09:32:29.662089 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0809 09:32:29.662093 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 09:32:29.664048 (+ 1955us) server_negotiation.cc:407] Connection header received
0809 09:32:29.665409 (+ 1361us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:32:29.665413 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:32:29.665510 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:32:29.665664 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:32:29.667302 (+ 1638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:29.668150 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:29.669941 (+ 1791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:29.670283 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:29.670366 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:29.674167 (+ 3801us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:32:29.674173 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:32:29.674181 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:32:29.674210 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:32:29.676048 (+ 1838us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:32:29.676578 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:32:29.676585 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:32:29.676588 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:32:29.676699 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:32:29.677093 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:32:29.677097 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:32:29.677098 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:32:29.677300 (+ 202us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:32:29.677386 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0809 09:32:29.677870 (+ 484us) server_negotiation.cc:299] Negotiation successful
0809 09:32:29.677980 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260809 09:32:29.678956 23266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:29.662359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:44501 (local address 127.20.125.131:37097)
0809 09:32:29.663873 (+ 1514us) negotiation.cc:107] Waiting for socket to connect
0809 09:32:29.663931 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0809 09:32:29.665123 (+ 1192us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:32:29.665895 (+ 772us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:32:29.665904 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:32:29.666374 (+ 470us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:32:29.667081 (+ 707us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:32:29.667101 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:32:29.668326 (+ 1225us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:32:29.668331 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:32:29.669787 (+ 1456us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:32:29.669797 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:32:29.670635 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:32:29.670642 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:32:29.670845 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:29.671572 (+ 727us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 09:32:29.671596 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:32:29.673952 (+ 2356us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 09:32:29.676163 (+ 2211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:32:29.676168 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:32:29.676178 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:32:29.676448 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:32:29.676804 (+ 356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:32:29.676807 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:32:29.676808 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:32:29.676950 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:32:29.677398 (+ 448us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 09:32:29.677403 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 09:32:29.677631 (+ 228us) client_negotiation.cc:769] Sending connection context
0809 09:32:29.677784 (+ 153us) client_negotiation.cc:240] Negotiation successful
0809 09:32:29.678049 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1203,"thread_start_us":153,"threads_started":1}
I20260809 09:32:29.679600 20982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu as pid 23227
I20260809 09:32:29.679757 20982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786267941944493-20982-0/minicluster-data/ts-2/wal/instance
I20260809 09:32:29.680068 20982 external_mini_cluster.cc:1428] Setting key 82071ee5c9300324842ca54556b99411
I20260809 09:32:29.680351 23432 heartbeater.cc:347] Connected to a master server at 127.20.125.190:44501
I20260809 09:32:29.680529 23432 heartbeater.cc:464] Registering TS with master...
I20260809 09:32:29.681164 23432 heartbeater.cc:511] Master 127.20.125.190:44501 requested a full tablet report, sending...
I20260809 09:32:29.683995 22666 ts_manager.cc:194] Registered new tserver with Master: f326fb02673b498d958b36e7fd8eb682 (127.20.125.131:42809)
I20260809 09:32:29.684762 22666 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.125.131@KRBTEST.COM'} at 127.20.125.131:37097
I20260809 09:32:29.690403 22740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:29.683525 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.131:48291 (local address 127.20.125.190:44501)
0809 09:32:29.683672 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0809 09:32:29.683676 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 09:32:29.683817 (+ 141us) server_negotiation.cc:407] Connection header received
0809 09:32:29.683940 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:32:29.683943 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:32:29.683981 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:32:29.684044 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:32:29.684553 (+ 509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:29.685211 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:29.686058 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:29.686363 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:29.686496 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:29.687695 (+ 1199us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:32:29.687698 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:32:29.687700 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:32:29.687724 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:32:29.689020 (+ 1296us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:32:29.689406 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:32:29.689408 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:32:29.689409 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:32:29.689483 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:32:29.689735 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:32:29.689737 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:32:29.689738 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:32:29.689891 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:32:29.689965 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0809 09:32:29.690210 (+ 245us) server_negotiation.cc:299] Negotiation successful
0809 09:32:29.690298 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260809 09:32:29.690408 23440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:29.683416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:44501 (local address 127.20.125.131:48291)
0809 09:32:29.683715 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0809 09:32:29.683728 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 09:32:29.683855 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:32:29.684092 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:32:29.684095 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:32:29.684318 (+ 223us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:32:29.684439 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:32:29.684447 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:32:29.685332 (+ 885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:32:29.685335 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:32:29.685939 (+ 604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:32:29.685948 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:32:29.686522 (+ 574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:32:29.686525 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:32:29.686617 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:29.687178 (+ 561us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 09:32:29.687199 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:32:29.687545 (+ 346us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 09:32:29.689120 (+ 1575us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:32:29.689123 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:32:29.689124 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:32:29.689319 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:32:29.689561 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:32:29.689563 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:32:29.689564 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:32:29.689640 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:32:29.689982 (+ 342us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 09:32:29.689985 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 09:32:29.690063 (+ 78us) client_negotiation.cc:769] Sending connection context
0809 09:32:29.690151 (+ 88us) client_negotiation.cc:240] Negotiation successful
0809 09:32:29.690256 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":214,"mutex_wait_us":3,"thread_start_us":95,"threads_started":1}
I20260809 09:32:29.694093 20982 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 09 09:32:29 dist-test-slave-7kzw krb5kdc[22595](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786267949, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 09:32:29 dist-test-slave-7kzw krb5kdc[22595](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786267949, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.20.125.190@KRBTEST.COM
I20260809 09:32:29.725920 22740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:29.717009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55604 (local address 127.20.125.190:44501)
0809 09:32:29.717144 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0809 09:32:29.717148 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 09:32:29.717161 (+ 13us) server_negotiation.cc:407] Connection header received
0809 09:32:29.717205 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:32:29.717207 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:32:29.717247 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:32:29.717363 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:32:29.718142 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:29.718871 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:29.719867 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:29.720105 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:29.720216 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:29.722865 (+ 2649us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:32:29.722869 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:32:29.722872 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:32:29.722909 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:32:29.724175 (+ 1266us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:32:29.724625 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:32:29.724628 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:32:29.724630 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:32:29.724719 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:32:29.725007 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:32:29.725011 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:32:29.725013 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:32:29.725204 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:32:29.725352 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0809 09:32:29.725620 (+ 268us) server_negotiation.cc:299] Negotiation successful
0809 09:32:29.725741 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260809 09:32:29.729378 22666 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55604:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 09:32:29.732594 22666 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 09:32:29.747117 23458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:29.741653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.131:42809 (local address 127.0.0.1:47000)
0809 09:32:29.741988 (+ 335us) negotiation.cc:107] Waiting for socket to connect
0809 09:32:29.742035 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0809 09:32:29.742161 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:32:29.742970 (+ 809us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:32:29.742975 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:32:29.743060 (+ 85us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 09:32:29.743208 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:32:29.743216 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:32:29.744296 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:32:29.744300 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:32:29.745315 (+ 1015us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:32:29.745347 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:32:29.746412 (+ 1065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:32:29.746415 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:32:29.746670 (+ 255us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:29.746721 (+ 51us) client_negotiation.cc:769] Sending connection context
0809 09:32:29.746829 (+ 108us) client_negotiation.cc:240] Negotiation successful
0809 09:32:29.746953 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":169,"mutex_wait_us":55,"thread_start_us":106,"threads_started":1}
I20260809 09:32:29.747118 23459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:29.741793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47000 (local address 127.20.125.131:42809)
0809 09:32:29.742101 (+ 308us) server_negotiation.cc:206] Beginning negotiation
0809 09:32:29.742115 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 09:32:29.742145 (+ 30us) server_negotiation.cc:407] Connection header received
0809 09:32:29.742346 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:32:29.742354 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:32:29.742608 (+ 254us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:32:29.742784 (+ 176us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 09:32:29.743348 (+ 564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:29.744159 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:29.745483 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:29.746293 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:29.746442 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:29.746624 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0809 09:32:29.746878 (+ 254us) server_negotiation.cc:299] Negotiation successful
0809 09:32:29.746970 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":207,"thread_start_us":122,"threads_started":1}
I20260809 09:32:29.748235 23461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:29.742393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.129:36447 (local address 127.0.0.1:38664)
0809 09:32:29.743324 (+ 931us) negotiation.cc:107] Waiting for socket to connect
0809 09:32:29.743338 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 09:32:29.743426 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:32:29.743935 (+ 509us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:32:29.743938 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:32:29.743961 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 09:32:29.744068 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:32:29.744076 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:32:29.745445 (+ 1369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:32:29.745449 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:32:29.746674 (+ 1225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:32:29.746690 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:32:29.747823 (+ 1133us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:32:29.747827 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:32:29.747910 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:29.747932 (+ 22us) client_negotiation.cc:769] Sending connection context
0809 09:32:29.748015 (+ 83us) client_negotiation.cc:240] Negotiation successful
0809 09:32:29.748086 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":872,"mutex_wait_us":47,"thread_start_us":130,"threads_started":1}
I20260809 09:32:29.748303 23462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:29.742580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38664 (local address 127.20.125.129:36447)
0809 09:32:29.743520 (+ 940us) server_negotiation.cc:206] Beginning negotiation
0809 09:32:29.743527 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 09:32:29.743548 (+ 21us) server_negotiation.cc:407] Connection header received
0809 09:32:29.743622 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:32:29.743627 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:32:29.743804 (+ 177us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:32:29.743954 (+ 150us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 09:32:29.744207 (+ 253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:29.745312 (+ 1105us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:29.746824 (+ 1512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:29.747698 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:29.747835 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:29.747960 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0809 09:32:29.748045 (+ 85us) server_negotiation.cc:299] Negotiation successful
0809 09:32:29.748130 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":832,"thread_start_us":128,"threads_started":1}
I20260809 09:32:29.749097 23466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:29.742743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.130:46859 (local address 127.0.0.1:33388)
0809 09:32:29.744172 (+ 1429us) negotiation.cc:107] Waiting for socket to connect
0809 09:32:29.744197 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0809 09:32:29.744282 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:32:29.744768 (+ 486us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:32:29.744771 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:32:29.744790 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 09:32:29.744879 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:32:29.744885 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:32:29.746222 (+ 1337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:32:29.746226 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:32:29.747350 (+ 1124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:32:29.747362 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:32:29.748731 (+ 1369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:32:29.748735 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:32:29.748820 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:29.748843 (+ 23us) client_negotiation.cc:769] Sending connection context
0809 09:32:29.748909 (+ 66us) client_negotiation.cc:240] Negotiation successful
0809 09:32:29.748977 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1357,"mutex_wait_us":3,"thread_start_us":87,"threads_started":1}
I20260809 09:32:29.749617 23468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:29.743038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33388 (local address 127.20.125.130:46859)
0809 09:32:29.743658 (+ 620us) server_negotiation.cc:206] Beginning negotiation
0809 09:32:29.743669 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 09:32:29.744372 (+ 703us) server_negotiation.cc:407] Connection header received
0809 09:32:29.744450 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:32:29.744455 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:32:29.744620 (+ 165us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:32:29.744760 (+ 140us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 09:32:29.744996 (+ 236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:29.746095 (+ 1099us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:29.747489 (+ 1394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:29.748286 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:29.748423 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:29.748535 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0809 09:32:29.749387 (+ 852us) server_negotiation.cc:299] Negotiation successful
0809 09:32:29.749482 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":528,"thread_start_us":172,"threads_started":1}
I20260809 09:32:29.750706 23305 tablet_service.cc:1467] Processing CreateTablet for tablet be3b6e7146034874b139ebf4ee567a5d (DEFAULT_TABLE table=test-table [id=922177ecda564121af97dd7ee72130d8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 09:32:29.750813 22875 tablet_service.cc:1467] Processing CreateTablet for tablet be3b6e7146034874b139ebf4ee567a5d (DEFAULT_TABLE table=test-table [id=922177ecda564121af97dd7ee72130d8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 09:32:29.751767 23305 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet be3b6e7146034874b139ebf4ee567a5d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 09:32:29.751839 22875 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet be3b6e7146034874b139ebf4ee567a5d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 09:32:29.752395 23112 tablet_service.cc:1467] Processing CreateTablet for tablet be3b6e7146034874b139ebf4ee567a5d (DEFAULT_TABLE table=test-table [id=922177ecda564121af97dd7ee72130d8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 09:32:29.753365 23112 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet be3b6e7146034874b139ebf4ee567a5d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 09:32:29.758798 23469 tablet_bootstrap.cc:492] T be3b6e7146034874b139ebf4ee567a5d P f326fb02673b498d958b36e7fd8eb682: Bootstrap starting.
I20260809 09:32:29.759889 23470 tablet_bootstrap.cc:492] T be3b6e7146034874b139ebf4ee567a5d P c405fafe9cca4eeea6ff3e56a6f8f61a: Bootstrap starting.
I20260809 09:32:29.761056 23469 tablet_bootstrap.cc:654] T be3b6e7146034874b139ebf4ee567a5d P f326fb02673b498d958b36e7fd8eb682: Neither blocks nor log segments found. Creating new log.
I20260809 09:32:29.761665 23471 tablet_bootstrap.cc:492] T be3b6e7146034874b139ebf4ee567a5d P 41c5693686644259835b5ee95bca2fad: Bootstrap starting.
I20260809 09:32:29.762036 23469 log.cc:824] T be3b6e7146034874b139ebf4ee567a5d P f326fb02673b498d958b36e7fd8eb682: Log is configured to *not* fsync() on all Append() calls
I20260809 09:32:29.762087 23470 tablet_bootstrap.cc:654] T be3b6e7146034874b139ebf4ee567a5d P c405fafe9cca4eeea6ff3e56a6f8f61a: Neither blocks nor log segments found. Creating new log.
I20260809 09:32:29.762933 23470 log.cc:824] T be3b6e7146034874b139ebf4ee567a5d P c405fafe9cca4eeea6ff3e56a6f8f61a: Log is configured to *not* fsync() on all Append() calls
I20260809 09:32:29.764139 23471 tablet_bootstrap.cc:654] T be3b6e7146034874b139ebf4ee567a5d P 41c5693686644259835b5ee95bca2fad: Neither blocks nor log segments found. Creating new log.
I20260809 09:32:29.764346 23469 tablet_bootstrap.cc:492] T be3b6e7146034874b139ebf4ee567a5d P f326fb02673b498d958b36e7fd8eb682: No bootstrap required, opened a new log
I20260809 09:32:29.764554 23469 ts_tablet_manager.cc:1397] T be3b6e7146034874b139ebf4ee567a5d P f326fb02673b498d958b36e7fd8eb682: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260809 09:32:29.764910 23470 tablet_bootstrap.cc:492] T be3b6e7146034874b139ebf4ee567a5d P c405fafe9cca4eeea6ff3e56a6f8f61a: No bootstrap required, opened a new log
I20260809 09:32:29.765019 23471 log.cc:824] T be3b6e7146034874b139ebf4ee567a5d P 41c5693686644259835b5ee95bca2fad: Log is configured to *not* fsync() on all Append() calls
I20260809 09:32:29.765093 23470 ts_tablet_manager.cc:1397] T be3b6e7146034874b139ebf4ee567a5d P c405fafe9cca4eeea6ff3e56a6f8f61a: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260809 09:32:29.766687 23471 tablet_bootstrap.cc:492] T be3b6e7146034874b139ebf4ee567a5d P 41c5693686644259835b5ee95bca2fad: No bootstrap required, opened a new log
I20260809 09:32:29.766880 23471 ts_tablet_manager.cc:1397] T be3b6e7146034874b139ebf4ee567a5d P 41c5693686644259835b5ee95bca2fad: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260809 09:32:29.768767 23469 raft_consensus.cc:348] T be3b6e7146034874b139ebf4ee567a5d P f326fb02673b498d958b36e7fd8eb682 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f326fb02673b498d958b36e7fd8eb682" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 42809 } } peers { permanent_uuid: "c405fafe9cca4eeea6ff3e56a6f8f61a" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 36447 } } peers { permanent_uuid: "41c5693686644259835b5ee95bca2fad" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 46859 } }
I20260809 09:32:29.769047 23469 raft_consensus.cc:374] T be3b6e7146034874b139ebf4ee567a5d P f326fb02673b498d958b36e7fd8eb682 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 09:32:29.769117 23469 raft_consensus.cc:729] T be3b6e7146034874b139ebf4ee567a5d P f326fb02673b498d958b36e7fd8eb682 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f326fb02673b498d958b36e7fd8eb682, State: Initialized, Role: FOLLOWER
I20260809 09:32:29.769575 23469 consensus_queue.cc:260] T be3b6e7146034874b139ebf4ee567a5d P f326fb02673b498d958b36e7fd8eb682 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f326fb02673b498d958b36e7fd8eb682" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 42809 } } peers { permanent_uuid: "c405fafe9cca4eeea6ff3e56a6f8f61a" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 36447 } } peers { permanent_uuid: "41c5693686644259835b5ee95bca2fad" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 46859 } }
I20260809 09:32:29.769518 23470 raft_consensus.cc:348] T be3b6e7146034874b139ebf4ee567a5d P c405fafe9cca4eeea6ff3e56a6f8f61a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f326fb02673b498d958b36e7fd8eb682" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 42809 } } peers { permanent_uuid: "c405fafe9cca4eeea6ff3e56a6f8f61a" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 36447 } } peers { permanent_uuid: "41c5693686644259835b5ee95bca2fad" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 46859 } }
I20260809 09:32:29.769763 23470 raft_consensus.cc:374] T be3b6e7146034874b139ebf4ee567a5d P c405fafe9cca4eeea6ff3e56a6f8f61a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 09:32:29.769825 23470 raft_consensus.cc:729] T be3b6e7146034874b139ebf4ee567a5d P c405fafe9cca4eeea6ff3e56a6f8f61a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c405fafe9cca4eeea6ff3e56a6f8f61a, State: Initialized, Role: FOLLOWER
I20260809 09:32:29.769773 23471 raft_consensus.cc:348] T be3b6e7146034874b139ebf4ee567a5d P 41c5693686644259835b5ee95bca2fad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f326fb02673b498d958b36e7fd8eb682" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 42809 } } peers { permanent_uuid: "c405fafe9cca4eeea6ff3e56a6f8f61a" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 36447 } } peers { permanent_uuid: "41c5693686644259835b5ee95bca2fad" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 46859 } }
I20260809 09:32:29.770005 23471 raft_consensus.cc:374] T be3b6e7146034874b139ebf4ee567a5d P 41c5693686644259835b5ee95bca2fad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 09:32:29.770056 23471 raft_consensus.cc:729] T be3b6e7146034874b139ebf4ee567a5d P 41c5693686644259835b5ee95bca2fad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41c5693686644259835b5ee95bca2fad, State: Initialized, Role: FOLLOWER
I20260809 09:32:29.770272 23470 consensus_queue.cc:260] T be3b6e7146034874b139ebf4ee567a5d P c405fafe9cca4eeea6ff3e56a6f8f61a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f326fb02673b498d958b36e7fd8eb682" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 42809 } } peers { permanent_uuid: "c405fafe9cca4eeea6ff3e56a6f8f61a" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 36447 } } peers { permanent_uuid: "41c5693686644259835b5ee95bca2fad" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 46859 } }
I20260809 09:32:29.770489 23471 consensus_queue.cc:260] T be3b6e7146034874b139ebf4ee567a5d P 41c5693686644259835b5ee95bca2fad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f326fb02673b498d958b36e7fd8eb682" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 42809 } } peers { permanent_uuid: "c405fafe9cca4eeea6ff3e56a6f8f61a" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 36447 } } peers { permanent_uuid: "41c5693686644259835b5ee95bca2fad" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 46859 } }
I20260809 09:32:29.770642 23469 ts_tablet_manager.cc:1428] T be3b6e7146034874b139ebf4ee567a5d P f326fb02673b498d958b36e7fd8eb682: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260809 09:32:29.771004 23432 heartbeater.cc:503] Master 127.20.125.190:44501 was elected leader, sending a full tablet report...
I20260809 09:32:29.771159 23470 ts_tablet_manager.cc:1428] T be3b6e7146034874b139ebf4ee567a5d P c405fafe9cca4eeea6ff3e56a6f8f61a: Time spent starting tablet: real 0.006s user 0.002s sys 0.005s
I20260809 09:32:29.771332 22986 heartbeater.cc:503] Master 127.20.125.190:44501 was elected leader, sending a full tablet report...
I20260809 09:32:29.772591 23471 ts_tablet_manager.cc:1428] T be3b6e7146034874b139ebf4ee567a5d P 41c5693686644259835b5ee95bca2fad: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260809 09:32:29.772644 23212 heartbeater.cc:503] Master 127.20.125.190:44501 was elected leader, sending a full tablet report...
W20260809 09:32:29.926164 23433 tablet.cc:2369] T be3b6e7146034874b139ebf4ee567a5d P f326fb02673b498d958b36e7fd8eb682: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 09:32:29.943696 23214 tablet.cc:2369] T be3b6e7146034874b139ebf4ee567a5d P 41c5693686644259835b5ee95bca2fad: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 09:32:29.950421 23476 raft_consensus.cc:482] T be3b6e7146034874b139ebf4ee567a5d P f326fb02673b498d958b36e7fd8eb682 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 09:32:29.950753 23476 raft_consensus.cc:504] T be3b6e7146034874b139ebf4ee567a5d P f326fb02673b498d958b36e7fd8eb682 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f326fb02673b498d958b36e7fd8eb682" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 42809 } } peers { permanent_uuid: "c405fafe9cca4eeea6ff3e56a6f8f61a" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 36447 } } peers { permanent_uuid: "41c5693686644259835b5ee95bca2fad" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 46859 } }
I20260809 09:32:29.952075 23476 leader_election.cc:288] T be3b6e7146034874b139ebf4ee567a5d P f326fb02673b498d958b36e7fd8eb682 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c405fafe9cca4eeea6ff3e56a6f8f61a (127.20.125.129:36447), 41c5693686644259835b5ee95bca2fad (127.20.125.130:46859)
I20260809 09:32:29.956581 23266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:29.952374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.129:36447 (local address 127.20.125.131:48251)
0809 09:32:29.952541 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0809 09:32:29.952558 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 09:32:29.952698 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:32:29.953006 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:32:29.953009 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:32:29.953030 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 09:32:29.953208 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:32:29.953221 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:32:29.954158 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:32:29.954161 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:32:29.955397 (+ 1236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:32:29.955408 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:32:29.956203 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:32:29.956207 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:32:29.956303 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:29.956325 (+ 22us) client_negotiation.cc:769] Sending connection context
0809 09:32:29.956400 (+ 75us) client_negotiation.cc:240] Negotiation successful
0809 09:32:29.956459 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":58,"mutex_wait_us":25}
I20260809 09:32:29.956656 23462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:29.952493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.131:48251 (local address 127.20.125.129:36447)
0809 09:32:29.952673 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0809 09:32:29.952678 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 09:32:29.952700 (+ 22us) server_negotiation.cc:407] Connection header received
0809 09:32:29.952819 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:32:29.952824 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:32:29.952890 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:32:29.952984 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 09:32:29.953369 (+ 385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:29.954041 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:29.955538 (+ 1497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:29.956057 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:29.956220 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:29.956281 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0809 09:32:29.956436 (+ 155us) server_negotiation.cc:299] Negotiation successful
0809 09:32:29.956516 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260809 09:32:29.957417 23468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:29.952870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.131:45899 (local address 127.20.125.130:46859)
0809 09:32:29.953107 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0809 09:32:29.953111 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 09:32:29.953306 (+ 195us) server_negotiation.cc:407] Connection header received
0809 09:32:29.953378 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:32:29.953381 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:32:29.953436 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:32:29.953533 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 09:32:29.954168 (+ 635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:29.954853 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:29.956087 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:29.956730 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:29.956870 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:29.956933 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0809 09:32:29.957218 (+ 285us) server_negotiation.cc:299] Negotiation successful
0809 09:32:29.957299 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138}
I20260809 09:32:29.957443 23479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:29.952705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.130:46859 (local address 127.20.125.131:45899)
0809 09:32:29.953091 (+ 386us) negotiation.cc:107] Waiting for socket to connect
0809 09:32:29.953105 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 09:32:29.953268 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:32:29.953546 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:32:29.953548 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:32:29.953562 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 09:32:29.954039 (+ 477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:32:29.954046 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:32:29.955001 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:32:29.955004 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:32:29.955958 (+ 954us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:32:29.955966 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:32:29.956955 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:32:29.956963 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:32:29.957064 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:29.957082 (+ 18us) client_negotiation.cc:769] Sending connection context
0809 09:32:29.957198 (+ 116us) client_negotiation.cc:240] Negotiation successful
0809 09:32:29.957273 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":326,"mutex_wait_us":4,"thread_start_us":155,"threads_started":1}
I20260809 09:32:29.957751 22906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "be3b6e7146034874b139ebf4ee567a5d" candidate_uuid: "f326fb02673b498d958b36e7fd8eb682" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c405fafe9cca4eeea6ff3e56a6f8f61a" is_pre_election: true
I20260809 09:32:29.958105 22906 raft_consensus.cc:2454] T be3b6e7146034874b139ebf4ee567a5d P c405fafe9cca4eeea6ff3e56a6f8f61a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f326fb02673b498d958b36e7fd8eb682 in term 0.
I20260809 09:32:29.958138 23141 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "be3b6e7146034874b139ebf4ee567a5d" candidate_uuid: "f326fb02673b498d958b36e7fd8eb682" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41c5693686644259835b5ee95bca2fad" is_pre_election: true
I20260809 09:32:29.958526 23141 raft_consensus.cc:2454] T be3b6e7146034874b139ebf4ee567a5d P 41c5693686644259835b5ee95bca2fad [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f326fb02673b498d958b36e7fd8eb682 in term 0.
I20260809 09:32:29.958703 23256 leader_election.cc:302] T be3b6e7146034874b139ebf4ee567a5d P f326fb02673b498d958b36e7fd8eb682 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c405fafe9cca4eeea6ff3e56a6f8f61a, f326fb02673b498d958b36e7fd8eb682; no voters:
I20260809 09:32:29.959054 23476 raft_consensus.cc:2781] T be3b6e7146034874b139ebf4ee567a5d P f326fb02673b498d958b36e7fd8eb682 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 09:32:29.959149 23476 raft_consensus.cc:482] T be3b6e7146034874b139ebf4ee567a5d P f326fb02673b498d958b36e7fd8eb682 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 09:32:29.959224 23476 raft_consensus.cc:3037] T be3b6e7146034874b139ebf4ee567a5d P f326fb02673b498d958b36e7fd8eb682 [term 0 FOLLOWER]: Advancing to term 1
I20260809 09:32:29.960397 23476 raft_consensus.cc:504] T be3b6e7146034874b139ebf4ee567a5d P f326fb02673b498d958b36e7fd8eb682 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f326fb02673b498d958b36e7fd8eb682" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 42809 } } peers { permanent_uuid: "c405fafe9cca4eeea6ff3e56a6f8f61a" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 36447 } } peers { permanent_uuid: "41c5693686644259835b5ee95bca2fad" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 46859 } }
I20260809 09:32:29.960780 23476 leader_election.cc:288] T be3b6e7146034874b139ebf4ee567a5d P f326fb02673b498d958b36e7fd8eb682 [CANDIDATE]: Term 1 election: Requested vote from peers c405fafe9cca4eeea6ff3e56a6f8f61a (127.20.125.129:36447), 41c5693686644259835b5ee95bca2fad (127.20.125.130:46859)
I20260809 09:32:29.961313 22906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "be3b6e7146034874b139ebf4ee567a5d" candidate_uuid: "f326fb02673b498d958b36e7fd8eb682" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c405fafe9cca4eeea6ff3e56a6f8f61a"
I20260809 09:32:29.961395 23141 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "be3b6e7146034874b139ebf4ee567a5d" candidate_uuid: "f326fb02673b498d958b36e7fd8eb682" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41c5693686644259835b5ee95bca2fad"
I20260809 09:32:29.961479 22906 raft_consensus.cc:3037] T be3b6e7146034874b139ebf4ee567a5d P c405fafe9cca4eeea6ff3e56a6f8f61a [term 0 FOLLOWER]: Advancing to term 1
I20260809 09:32:29.961539 23141 raft_consensus.cc:3037] T be3b6e7146034874b139ebf4ee567a5d P 41c5693686644259835b5ee95bca2fad [term 0 FOLLOWER]: Advancing to term 1
I20260809 09:32:29.962996 23141 raft_consensus.cc:2454] T be3b6e7146034874b139ebf4ee567a5d P 41c5693686644259835b5ee95bca2fad [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f326fb02673b498d958b36e7fd8eb682 in term 1.
W20260809 09:32:29.963325 22988 tablet.cc:2369] T be3b6e7146034874b139ebf4ee567a5d P c405fafe9cca4eeea6ff3e56a6f8f61a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 09:32:29.963352 23256 leader_election.cc:302] T be3b6e7146034874b139ebf4ee567a5d P f326fb02673b498d958b36e7fd8eb682 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41c5693686644259835b5ee95bca2fad, f326fb02673b498d958b36e7fd8eb682; no voters:
I20260809 09:32:29.963523 23476 raft_consensus.cc:2781] T be3b6e7146034874b139ebf4ee567a5d P f326fb02673b498d958b36e7fd8eb682 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 09:32:29.963821 23476 raft_consensus.cc:686] T be3b6e7146034874b139ebf4ee567a5d P f326fb02673b498d958b36e7fd8eb682 [term 1 LEADER]: Becoming Leader. State: Replica: f326fb02673b498d958b36e7fd8eb682, State: Running, Role: LEADER
I20260809 09:32:29.963109 22906 raft_consensus.cc:2454] T be3b6e7146034874b139ebf4ee567a5d P c405fafe9cca4eeea6ff3e56a6f8f61a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f326fb02673b498d958b36e7fd8eb682 in term 1.
I20260809 09:32:29.964186 23476 consensus_queue.cc:237] T be3b6e7146034874b139ebf4ee567a5d P f326fb02673b498d958b36e7fd8eb682 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f326fb02673b498d958b36e7fd8eb682" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 42809 } } peers { permanent_uuid: "c405fafe9cca4eeea6ff3e56a6f8f61a" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 36447 } } peers { permanent_uuid: "41c5693686644259835b5ee95bca2fad" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 46859 } }
I20260809 09:32:29.967510 22664 catalog_manager.cc:5495] T be3b6e7146034874b139ebf4ee567a5d P f326fb02673b498d958b36e7fd8eb682 reported cstate change: term changed from 0 to 1, leader changed from <none> to f326fb02673b498d958b36e7fd8eb682 (127.20.125.131). New cstate: current_term: 1 leader_uuid: "f326fb02673b498d958b36e7fd8eb682" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f326fb02673b498d958b36e7fd8eb682" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 42809 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c405fafe9cca4eeea6ff3e56a6f8f61a" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 36447 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "41c5693686644259835b5ee95bca2fad" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 46859 } health_report { overall_health: UNKNOWN } } }
I20260809 09:32:29.991025 23459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:29.987227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47004 (local address 127.20.125.131:42809)
0809 09:32:29.987372 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0809 09:32:29.987383 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 09:32:29.987398 (+ 15us) server_negotiation.cc:407] Connection header received
0809 09:32:29.987615 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:32:29.987617 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:32:29.987673 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:32:29.987768 (+ 95us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 09:32:29.988017 (+ 249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:29.988536 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:29.989481 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:29.989693 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:29.989777 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:29.990101 (+ 324us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 09:32:29.990629 (+ 528us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 09:32:29.990721 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0809 09:32:29.990859 (+ 138us) server_negotiation.cc:299] Negotiation successful
0809 09:32:29.990914 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260809 09:32:29.997260 23141 raft_consensus.cc:1261] T be3b6e7146034874b139ebf4ee567a5d P 41c5693686644259835b5ee95bca2fad [term 1 FOLLOWER]: Refusing update from remote peer f326fb02673b498d958b36e7fd8eb682: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 09:32:29.997282 22906 raft_consensus.cc:1261] T be3b6e7146034874b139ebf4ee567a5d P c405fafe9cca4eeea6ff3e56a6f8f61a [term 1 FOLLOWER]: Refusing update from remote peer f326fb02673b498d958b36e7fd8eb682: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 09:32:29.998027 23480 consensus_queue.cc:1035] T be3b6e7146034874b139ebf4ee567a5d P f326fb02673b498d958b36e7fd8eb682 [LEADER]: Connected to new peer: Peer: permanent_uuid: "41c5693686644259835b5ee95bca2fad" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 46859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 09:32:29.998328 23476 consensus_queue.cc:1035] T be3b6e7146034874b139ebf4ee567a5d P f326fb02673b498d958b36e7fd8eb682 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c405fafe9cca4eeea6ff3e56a6f8f61a" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 36447 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 09:32:30.007993 23491 mvcc.cc:205] Tried to move back new op lower bound from 7316553523181232128 to 7316553523056279552. Current Snapshot: MvccSnapshot[applied={T|T < 7316553523181232128}]
I20260809 09:32:30.021826 22664 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55604:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 09:32:30.022073 22664 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55604:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 09:32:30.025144 22664 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0ef673811e7443ac84e6e78369057d13: Sending DeleteTablet for 3 replicas of tablet be3b6e7146034874b139ebf4ee567a5d
I20260809 09:32:30.025995 23305 tablet_service.cc:1514] Processing DeleteTablet for tablet be3b6e7146034874b139ebf4ee567a5d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 09:32:30 UTC) from {username='kudu', principal='kudu/127.20.125.190@KRBTEST.COM'} at 127.0.0.1:47000
I20260809 09:32:30.026361 23112 tablet_service.cc:1514] Processing DeleteTablet for tablet be3b6e7146034874b139ebf4ee567a5d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 09:32:30 UTC) from {username='kudu', principal='kudu/127.20.125.190@KRBTEST.COM'} at 127.0.0.1:33388
I20260809 09:32:30.026659 22875 tablet_service.cc:1514] Processing DeleteTablet for tablet be3b6e7146034874b139ebf4ee567a5d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 09:32:30 UTC) from {username='kudu', principal='kudu/127.20.125.190@KRBTEST.COM'} at 127.0.0.1:38664
I20260809 09:32:30.026896 23503 tablet_replica.cc:330] T be3b6e7146034874b139ebf4ee567a5d P f326fb02673b498d958b36e7fd8eb682: stopping tablet replica
I20260809 09:32:30.027209 23505 tablet_replica.cc:330] T be3b6e7146034874b139ebf4ee567a5d P c405fafe9cca4eeea6ff3e56a6f8f61a: stopping tablet replica
I20260809 09:32:30.027408 23503 raft_consensus.cc:2229] T be3b6e7146034874b139ebf4ee567a5d P f326fb02673b498d958b36e7fd8eb682 [term 1 LEADER]: Raft consensus shutting down.
I20260809 09:32:30.027570 23505 raft_consensus.cc:2229] T be3b6e7146034874b139ebf4ee567a5d P c405fafe9cca4eeea6ff3e56a6f8f61a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 09:32:30.027815 22664 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55604:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 09:32:30.027944 23505 raft_consensus.cc:2258] T be3b6e7146034874b139ebf4ee567a5d P c405fafe9cca4eeea6ff3e56a6f8f61a [term 1 FOLLOWER]: Raft consensus is shut down!
W20260809 09:32:30.028296 22664 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 09:32:30.028350 23504 tablet_replica.cc:330] T be3b6e7146034874b139ebf4ee567a5d P 41c5693686644259835b5ee95bca2fad: stopping tablet replica
I20260809 09:32:30.028329 23503 raft_consensus.cc:2258] T be3b6e7146034874b139ebf4ee567a5d P f326fb02673b498d958b36e7fd8eb682 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 09:32:30.028662 23504 raft_consensus.cc:2229] T be3b6e7146034874b139ebf4ee567a5d P 41c5693686644259835b5ee95bca2fad [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 09:32:30.028987 23504 raft_consensus.cc:2258] T be3b6e7146034874b139ebf4ee567a5d P 41c5693686644259835b5ee95bca2fad [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 09:32:30.030387 23503 ts_tablet_manager.cc:1905] T be3b6e7146034874b139ebf4ee567a5d P f326fb02673b498d958b36e7fd8eb682: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 09:32:30.030503 23505 ts_tablet_manager.cc:1905] T be3b6e7146034874b139ebf4ee567a5d P c405fafe9cca4eeea6ff3e56a6f8f61a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 09:32:30.033776 23505 ts_tablet_manager.cc:1918] T be3b6e7146034874b139ebf4ee567a5d P c405fafe9cca4eeea6ff3e56a6f8f61a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 09:32:30.033891 23505 log.cc:1196] T be3b6e7146034874b139ebf4ee567a5d P c405fafe9cca4eeea6ff3e56a6f8f61a: Deleting WAL directory at /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786267941944493-20982-0/minicluster-data/ts-0/wal/wals/be3b6e7146034874b139ebf4ee567a5d
I20260809 09:32:30.034219 23505 ts_tablet_manager.cc:1939] T be3b6e7146034874b139ebf4ee567a5d P c405fafe9cca4eeea6ff3e56a6f8f61a: Deleting consensus metadata
I20260809 09:32:30.035874 23504 ts_tablet_manager.cc:1905] T be3b6e7146034874b139ebf4ee567a5d P 41c5693686644259835b5ee95bca2fad: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 09:32:30.035913 22875 tablet_service.cc:1467] Processing CreateTablet for tablet 6e8ecb32d2e94f7cb93eb8fc7b6e5f4e (DEFAULT_TABLE table=test-table [id=aeab29f88d714c90ac05ecb005b5b550]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 09:32:30.036224 22875 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6e8ecb32d2e94f7cb93eb8fc7b6e5f4e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 09:32:30.037256 23503 ts_tablet_manager.cc:1918] T be3b6e7146034874b139ebf4ee567a5d P f326fb02673b498d958b36e7fd8eb682: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 09:32:30.037395 23503 log.cc:1196] T be3b6e7146034874b139ebf4ee567a5d P f326fb02673b498d958b36e7fd8eb682: Deleting WAL directory at /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786267941944493-20982-0/minicluster-data/ts-2/wal/wals/be3b6e7146034874b139ebf4ee567a5d
I20260809 09:32:30.037887 23503 ts_tablet_manager.cc:1939] T be3b6e7146034874b139ebf4ee567a5d P f326fb02673b498d958b36e7fd8eb682: Deleting consensus metadata
I20260809 09:32:30.038089 23305 tablet_service.cc:1467] Processing CreateTablet for tablet 6e8ecb32d2e94f7cb93eb8fc7b6e5f4e (DEFAULT_TABLE table=test-table [id=aeab29f88d714c90ac05ecb005b5b550]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 09:32:30.038635 23504 ts_tablet_manager.cc:1918] T be3b6e7146034874b139ebf4ee567a5d P 41c5693686644259835b5ee95bca2fad: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 09:32:30.038743 23504 log.cc:1196] T be3b6e7146034874b139ebf4ee567a5d P 41c5693686644259835b5ee95bca2fad: Deleting WAL directory at /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786267941944493-20982-0/minicluster-data/ts-1/wal/wals/be3b6e7146034874b139ebf4ee567a5d
I20260809 09:32:30.038807 23112 tablet_service.cc:1467] Processing CreateTablet for tablet 6e8ecb32d2e94f7cb93eb8fc7b6e5f4e (DEFAULT_TABLE table=test-table [id=aeab29f88d714c90ac05ecb005b5b550]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 09:32:30.039036 23504 ts_tablet_manager.cc:1939] T be3b6e7146034874b139ebf4ee567a5d P 41c5693686644259835b5ee95bca2fad: Deleting consensus metadata
I20260809 09:32:30.039125 23112 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6e8ecb32d2e94f7cb93eb8fc7b6e5f4e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 09:32:30.040825 23305 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6e8ecb32d2e94f7cb93eb8fc7b6e5f4e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 09:32:30.041916 23471 tablet_bootstrap.cc:492] T 6e8ecb32d2e94f7cb93eb8fc7b6e5f4e P 41c5693686644259835b5ee95bca2fad: Bootstrap starting.
I20260809 09:32:30.042210 22639 catalog_manager.cc:4841] TS c405fafe9cca4eeea6ff3e56a6f8f61a (127.20.125.129:36447): tablet be3b6e7146034874b139ebf4ee567a5d (table test-table [id=922177ecda564121af97dd7ee72130d8]) successfully deleted
I20260809 09:32:30.042424 22639 catalog_manager.cc:4841] TS 41c5693686644259835b5ee95bca2fad (127.20.125.130:46859): tablet be3b6e7146034874b139ebf4ee567a5d (table test-table [id=922177ecda564121af97dd7ee72130d8]) successfully deleted
I20260809 09:32:30.042656 22639 catalog_manager.cc:4841] TS f326fb02673b498d958b36e7fd8eb682 (127.20.125.131:42809): tablet be3b6e7146034874b139ebf4ee567a5d (table test-table [id=922177ecda564121af97dd7ee72130d8]) successfully deleted
I20260809 09:32:30.042928 23471 tablet_bootstrap.cc:654] T 6e8ecb32d2e94f7cb93eb8fc7b6e5f4e P 41c5693686644259835b5ee95bca2fad: Neither blocks nor log segments found. Creating new log.
I20260809 09:32:30.043387 23469 tablet_bootstrap.cc:492] T 6e8ecb32d2e94f7cb93eb8fc7b6e5f4e P f326fb02673b498d958b36e7fd8eb682: Bootstrap starting.
I20260809 09:32:30.044407 23469 tablet_bootstrap.cc:654] T 6e8ecb32d2e94f7cb93eb8fc7b6e5f4e P f326fb02673b498d958b36e7fd8eb682: Neither blocks nor log segments found. Creating new log.
I20260809 09:32:30.044492 23471 tablet_bootstrap.cc:492] T 6e8ecb32d2e94f7cb93eb8fc7b6e5f4e P 41c5693686644259835b5ee95bca2fad: No bootstrap required, opened a new log
I20260809 09:32:30.044567 23471 ts_tablet_manager.cc:1397] T 6e8ecb32d2e94f7cb93eb8fc7b6e5f4e P 41c5693686644259835b5ee95bca2fad: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 09:32:30.044997 23471 raft_consensus.cc:348] T 6e8ecb32d2e94f7cb93eb8fc7b6e5f4e P 41c5693686644259835b5ee95bca2fad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c405fafe9cca4eeea6ff3e56a6f8f61a" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 36447 } } peers { permanent_uuid: "41c5693686644259835b5ee95bca2fad" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 46859 } } peers { permanent_uuid: "f326fb02673b498d958b36e7fd8eb682" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 42809 } }
I20260809 09:32:30.045104 23471 raft_consensus.cc:374] T 6e8ecb32d2e94f7cb93eb8fc7b6e5f4e P 41c5693686644259835b5ee95bca2fad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 09:32:30.045133 23471 raft_consensus.cc:729] T 6e8ecb32d2e94f7cb93eb8fc7b6e5f4e P 41c5693686644259835b5ee95bca2fad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41c5693686644259835b5ee95bca2fad, State: Initialized, Role: FOLLOWER
I20260809 09:32:30.045261 23471 consensus_queue.cc:260] T 6e8ecb32d2e94f7cb93eb8fc7b6e5f4e P 41c5693686644259835b5ee95bca2fad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c405fafe9cca4eeea6ff3e56a6f8f61a" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 36447 } } peers { permanent_uuid: "41c5693686644259835b5ee95bca2fad" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 46859 } } peers { permanent_uuid: "f326fb02673b498d958b36e7fd8eb682" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 42809 } }
I20260809 09:32:30.045502 23471 ts_tablet_manager.cc:1428] T 6e8ecb32d2e94f7cb93eb8fc7b6e5f4e P 41c5693686644259835b5ee95bca2fad: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260809 09:32:30.046032 23469 tablet_bootstrap.cc:492] T 6e8ecb32d2e94f7cb93eb8fc7b6e5f4e P f326fb02673b498d958b36e7fd8eb682: No bootstrap required, opened a new log
I20260809 09:32:30.046072 23470 tablet_bootstrap.cc:492] T 6e8ecb32d2e94f7cb93eb8fc7b6e5f4e P c405fafe9cca4eeea6ff3e56a6f8f61a: Bootstrap starting.
I20260809 09:32:30.046109 23469 ts_tablet_manager.cc:1397] T 6e8ecb32d2e94f7cb93eb8fc7b6e5f4e P f326fb02673b498d958b36e7fd8eb682: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 09:32:30.046625 23469 raft_consensus.cc:348] T 6e8ecb32d2e94f7cb93eb8fc7b6e5f4e P f326fb02673b498d958b36e7fd8eb682 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c405fafe9cca4eeea6ff3e56a6f8f61a" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 36447 } } peers { permanent_uuid: "41c5693686644259835b5ee95bca2fad" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 46859 } } peers { permanent_uuid: "f326fb02673b498d958b36e7fd8eb682" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 42809 } }
I20260809 09:32:30.046763 23469 raft_consensus.cc:374] T 6e8ecb32d2e94f7cb93eb8fc7b6e5f4e P f326fb02673b498d958b36e7fd8eb682 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 09:32:30.046802 23469 raft_consensus.cc:729] T 6e8ecb32d2e94f7cb93eb8fc7b6e5f4e P f326fb02673b498d958b36e7fd8eb682 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f326fb02673b498d958b36e7fd8eb682, State: Initialized, Role: FOLLOWER
I20260809 09:32:30.046937 23469 consensus_queue.cc:260] T 6e8ecb32d2e94f7cb93eb8fc7b6e5f4e P f326fb02673b498d958b36e7fd8eb682 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c405fafe9cca4eeea6ff3e56a6f8f61a" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 36447 } } peers { permanent_uuid: "41c5693686644259835b5ee95bca2fad" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 46859 } } peers { permanent_uuid: "f326fb02673b498d958b36e7fd8eb682" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 42809 } }
I20260809 09:32:30.047084 23470 tablet_bootstrap.cc:654] T 6e8ecb32d2e94f7cb93eb8fc7b6e5f4e P c405fafe9cca4eeea6ff3e56a6f8f61a: Neither blocks nor log segments found. Creating new log.
I20260809 09:32:30.047250 23469 ts_tablet_manager.cc:1428] T 6e8ecb32d2e94f7cb93eb8fc7b6e5f4e P f326fb02673b498d958b36e7fd8eb682: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260809 09:32:30.048336 23470 tablet_bootstrap.cc:492] T 6e8ecb32d2e94f7cb93eb8fc7b6e5f4e P c405fafe9cca4eeea6ff3e56a6f8f61a: No bootstrap required, opened a new log
I20260809 09:32:30.048437 23470 ts_tablet_manager.cc:1397] T 6e8ecb32d2e94f7cb93eb8fc7b6e5f4e P c405fafe9cca4eeea6ff3e56a6f8f61a: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 09:32:30.049310 23470 raft_consensus.cc:348] T 6e8ecb32d2e94f7cb93eb8fc7b6e5f4e P c405fafe9cca4eeea6ff3e56a6f8f61a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c405fafe9cca4eeea6ff3e56a6f8f61a" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 36447 } } peers { permanent_uuid: "41c5693686644259835b5ee95bca2fad" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 46859 } } peers { permanent_uuid: "f326fb02673b498d958b36e7fd8eb682" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 42809 } }
I20260809 09:32:30.049433 23470 raft_consensus.cc:374] T 6e8ecb32d2e94f7cb93eb8fc7b6e5f4e P c405fafe9cca4eeea6ff3e56a6f8f61a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 09:32:30.049479 23470 raft_consensus.cc:729] T 6e8ecb32d2e94f7cb93eb8fc7b6e5f4e P c405fafe9cca4eeea6ff3e56a6f8f61a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c405fafe9cca4eeea6ff3e56a6f8f61a, State: Initialized, Role: FOLLOWER
I20260809 09:32:30.049611 23470 consensus_queue.cc:260] T 6e8ecb32d2e94f7cb93eb8fc7b6e5f4e P c405fafe9cca4eeea6ff3e56a6f8f61a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c405fafe9cca4eeea6ff3e56a6f8f61a" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 36447 } } peers { permanent_uuid: "41c5693686644259835b5ee95bca2fad" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 46859 } } peers { permanent_uuid: "f326fb02673b498d958b36e7fd8eb682" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 42809 } }
I20260809 09:32:30.049909 23470 ts_tablet_manager.cc:1428] T 6e8ecb32d2e94f7cb93eb8fc7b6e5f4e P c405fafe9cca4eeea6ff3e56a6f8f61a: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260809 09:32:30.136441 23477 raft_consensus.cc:482] T 6e8ecb32d2e94f7cb93eb8fc7b6e5f4e P c405fafe9cca4eeea6ff3e56a6f8f61a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 09:32:30.136672 23477 raft_consensus.cc:504] T 6e8ecb32d2e94f7cb93eb8fc7b6e5f4e P c405fafe9cca4eeea6ff3e56a6f8f61a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c405fafe9cca4eeea6ff3e56a6f8f61a" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 36447 } } peers { permanent_uuid: "41c5693686644259835b5ee95bca2fad" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 46859 } } peers { permanent_uuid: "f326fb02673b498d958b36e7fd8eb682" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 42809 } }
I20260809 09:32:30.138132 23477 leader_election.cc:288] T 6e8ecb32d2e94f7cb93eb8fc7b6e5f4e P c405fafe9cca4eeea6ff3e56a6f8f61a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 41c5693686644259835b5ee95bca2fad (127.20.125.130:46859), f326fb02673b498d958b36e7fd8eb682 (127.20.125.131:42809)
I20260809 09:32:30.143697 23513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:30.138946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.131:42809 (local address 127.20.125.129:44309)
0809 09:32:30.139947 (+ 1001us) negotiation.cc:107] Waiting for socket to connect
0809 09:32:30.139959 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 09:32:30.140083 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:32:30.140341 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:32:30.140345 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:32:30.140364 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 09:32:30.140486 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:32:30.140493 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:32:30.141406 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:32:30.141410 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:32:30.142221 (+ 811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:32:30.142231 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:32:30.143209 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:32:30.143213 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:32:30.143323 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:30.143352 (+ 29us) client_negotiation.cc:769] Sending connection context
0809 09:32:30.143447 (+ 95us) client_negotiation.cc:240] Negotiation successful
0809 09:32:30.143523 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":942,"thread_start_us":271,"threads_started":1}
I20260809 09:32:30.143896 23459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:30.139654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.129:44309 (local address 127.20.125.131:42809)
0809 09:32:30.139806 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0809 09:32:30.139811 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 09:32:30.140045 (+ 234us) server_negotiation.cc:407] Connection header received
0809 09:32:30.140194 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:32:30.140197 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:32:30.140254 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:32:30.140328 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 09:32:30.140613 (+ 285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:30.141289 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:30.142354 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:30.142993 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:30.143138 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:30.143197 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0809 09:32:30.143638 (+ 441us) server_negotiation.cc:299] Negotiation successful
0809 09:32:30.143724 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260809 09:32:30.144387 23468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:30.138707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.129:33711 (local address 127.20.125.130:46859)
0809 09:32:30.139480 (+ 773us) server_negotiation.cc:206] Beginning negotiation
0809 09:32:30.139486 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 09:32:30.139512 (+ 26us) server_negotiation.cc:407] Connection header received
0809 09:32:30.139572 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:32:30.139575 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:32:30.139624 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:32:30.139703 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 09:32:30.140260 (+ 557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:30.141040 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:30.142606 (+ 1566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:30.143198 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:30.143311 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:30.143361 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0809 09:32:30.144179 (+ 818us) server_negotiation.cc:299] Negotiation successful
0809 09:32:30.144266 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":646}
I20260809 09:32:30.144461 23512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:30.138593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.130:46859 (local address 127.20.125.129:33711)
0809 09:32:30.138967 (+ 374us) negotiation.cc:107] Waiting for socket to connect
0809 09:32:30.139013 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 09:32:30.139381 (+ 368us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:32:30.139893 (+ 512us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:32:30.139896 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:32:30.139922 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 09:32:30.140061 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:32:30.140070 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:32:30.141175 (+ 1105us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:32:30.141179 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:32:30.142407 (+ 1228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:32:30.142419 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:32:30.143937 (+ 1518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:32:30.143943 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:32:30.144047 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:30.144072 (+ 25us) client_negotiation.cc:769] Sending connection context
0809 09:32:30.144159 (+ 87us) client_negotiation.cc:240] Negotiation successful
0809 09:32:30.144317 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":271,"thread_start_us":78,"threads_started":1}
I20260809 09:32:30.144639 23329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e8ecb32d2e94f7cb93eb8fc7b6e5f4e" candidate_uuid: "c405fafe9cca4eeea6ff3e56a6f8f61a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f326fb02673b498d958b36e7fd8eb682" is_pre_election: true
I20260809 09:32:30.144951 23329 raft_consensus.cc:2454] T 6e8ecb32d2e94f7cb93eb8fc7b6e5f4e P f326fb02673b498d958b36e7fd8eb682 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c405fafe9cca4eeea6ff3e56a6f8f61a in term 0.
I20260809 09:32:30.145452 22794 leader_election.cc:302] T 6e8ecb32d2e94f7cb93eb8fc7b6e5f4e P c405fafe9cca4eeea6ff3e56a6f8f61a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c405fafe9cca4eeea6ff3e56a6f8f61a, f326fb02673b498d958b36e7fd8eb682; no voters:
I20260809 09:32:30.145526 23141 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e8ecb32d2e94f7cb93eb8fc7b6e5f4e" candidate_uuid: "c405fafe9cca4eeea6ff3e56a6f8f61a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41c5693686644259835b5ee95bca2fad" is_pre_election: true
I20260809 09:32:30.145669 23141 raft_consensus.cc:2454] T 6e8ecb32d2e94f7cb93eb8fc7b6e5f4e P 41c5693686644259835b5ee95bca2fad [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c405fafe9cca4eeea6ff3e56a6f8f61a in term 0.
I20260809 09:32:30.145750 23477 raft_consensus.cc:2781] T 6e8ecb32d2e94f7cb93eb8fc7b6e5f4e P c405fafe9cca4eeea6ff3e56a6f8f61a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 09:32:30.145813 23477 raft_consensus.cc:482] T 6e8ecb32d2e94f7cb93eb8fc7b6e5f4e P c405fafe9cca4eeea6ff3e56a6f8f61a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 09:32:30.145864 23477 raft_consensus.cc:3037] T 6e8ecb32d2e94f7cb93eb8fc7b6e5f4e P c405fafe9cca4eeea6ff3e56a6f8f61a [term 0 FOLLOWER]: Advancing to term 1
I20260809 09:32:30.146986 23477 raft_consensus.cc:504] T 6e8ecb32d2e94f7cb93eb8fc7b6e5f4e P c405fafe9cca4eeea6ff3e56a6f8f61a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c405fafe9cca4eeea6ff3e56a6f8f61a" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 36447 } } peers { permanent_uuid: "41c5693686644259835b5ee95bca2fad" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 46859 } } peers { permanent_uuid: "f326fb02673b498d958b36e7fd8eb682" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 42809 } }
I20260809 09:32:30.147539 23477 leader_election.cc:288] T 6e8ecb32d2e94f7cb93eb8fc7b6e5f4e P c405fafe9cca4eeea6ff3e56a6f8f61a [CANDIDATE]: Term 1 election: Requested vote from peers 41c5693686644259835b5ee95bca2fad (127.20.125.130:46859), f326fb02673b498d958b36e7fd8eb682 (127.20.125.131:42809)
I20260809 09:32:30.148208 23141 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e8ecb32d2e94f7cb93eb8fc7b6e5f4e" candidate_uuid: "c405fafe9cca4eeea6ff3e56a6f8f61a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41c5693686644259835b5ee95bca2fad"
I20260809 09:32:30.148372 23141 raft_consensus.cc:3037] T 6e8ecb32d2e94f7cb93eb8fc7b6e5f4e P 41c5693686644259835b5ee95bca2fad [term 0 FOLLOWER]: Advancing to term 1
I20260809 09:32:30.149518 23141 raft_consensus.cc:2454] T 6e8ecb32d2e94f7cb93eb8fc7b6e5f4e P 41c5693686644259835b5ee95bca2fad [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c405fafe9cca4eeea6ff3e56a6f8f61a in term 1.
I20260809 09:32:30.149806 23329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e8ecb32d2e94f7cb93eb8fc7b6e5f4e" candidate_uuid: "c405fafe9cca4eeea6ff3e56a6f8f61a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f326fb02673b498d958b36e7fd8eb682"
I20260809 09:32:30.149842 22794 leader_election.cc:302] T 6e8ecb32d2e94f7cb93eb8fc7b6e5f4e P c405fafe9cca4eeea6ff3e56a6f8f61a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41c5693686644259835b5ee95bca2fad, c405fafe9cca4eeea6ff3e56a6f8f61a; no voters:
I20260809 09:32:30.149968 23329 raft_consensus.cc:3037] T 6e8ecb32d2e94f7cb93eb8fc7b6e5f4e P f326fb02673b498d958b36e7fd8eb682 [term 0 FOLLOWER]: Advancing to term 1
I20260809 09:32:30.151106 23329 raft_consensus.cc:2454] T 6e8ecb32d2e94f7cb93eb8fc7b6e5f4e P f326fb02673b498d958b36e7fd8eb682 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c405fafe9cca4eeea6ff3e56a6f8f61a in term 1.
I20260809 09:32:30.151357 23477 raft_consensus.cc:2781] T 6e8ecb32d2e94f7cb93eb8fc7b6e5f4e P c405fafe9cca4eeea6ff3e56a6f8f61a [term 1 FOLLOWER]: Leader election won for term 1
I20260809 09:32:30.151655 23477 raft_consensus.cc:686] T 6e8ecb32d2e94f7cb93eb8fc7b6e5f4e P c405fafe9cca4eeea6ff3e56a6f8f61a [term 1 LEADER]: Becoming Leader. State: Replica: c405fafe9cca4eeea6ff3e56a6f8f61a, State: Running, Role: LEADER
I20260809 09:32:30.151955 23477 consensus_queue.cc:237] T 6e8ecb32d2e94f7cb93eb8fc7b6e5f4e P c405fafe9cca4eeea6ff3e56a6f8f61a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c405fafe9cca4eeea6ff3e56a6f8f61a" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 36447 } } peers { permanent_uuid: "41c5693686644259835b5ee95bca2fad" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 46859 } } peers { permanent_uuid: "f326fb02673b498d958b36e7fd8eb682" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 42809 } }
I20260809 09:32:30.154585 22664 catalog_manager.cc:5495] T 6e8ecb32d2e94f7cb93eb8fc7b6e5f4e P c405fafe9cca4eeea6ff3e56a6f8f61a reported cstate change: term changed from 0 to 1, leader changed from <none> to c405fafe9cca4eeea6ff3e56a6f8f61a (127.20.125.129). New cstate: current_term: 1 leader_uuid: "c405fafe9cca4eeea6ff3e56a6f8f61a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c405fafe9cca4eeea6ff3e56a6f8f61a" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 36447 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "41c5693686644259835b5ee95bca2fad" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 46859 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f326fb02673b498d958b36e7fd8eb682" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 42809 } health_report { overall_health: UNKNOWN } } }
Aug 09 09:32:30 dist-test-slave-7kzw krb5kdc[22595](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786267948, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.190@KRBTEST.COM for kudu/127.20.125.190@KRBTEST.COM
I20260809 09:32:30.202054 22740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:30.192488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55612 (local address 127.20.125.190:44501)
0809 09:32:30.192672 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0809 09:32:30.192677 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 09:32:30.192797 (+ 120us) server_negotiation.cc:407] Connection header received
0809 09:32:30.192907 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:32:30.192910 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:32:30.192954 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:32:30.193027 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:32:30.193784 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:30.194313 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:30.195271 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:30.195484 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:30.195667 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:30.198422 (+ 2755us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:32:30.198425 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:32:30.198427 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:32:30.198449 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:32:30.200248 (+ 1799us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:32:30.200824 (+ 576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:32:30.200827 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:32:30.200828 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:32:30.200914 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:32:30.201200 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:32:30.201203 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:32:30.201204 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:32:30.201392 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:32:30.201473 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0809 09:32:30.201802 (+ 329us) server_negotiation.cc:299] Negotiation successful
0809 09:32:30.201910 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260809 09:32:30.202059 23530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:30.192339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:44501 (local address 127.0.0.1:55612)
0809 09:32:30.192695 (+ 356us) negotiation.cc:107] Waiting for socket to connect
0809 09:32:30.192709 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 09:32:30.192823 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:32:30.193084 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:32:30.193088 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:32:30.193514 (+ 426us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:32:30.193653 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:32:30.193660 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:32:30.194528 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:32:30.194535 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:32:30.195124 (+ 589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:32:30.195133 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:32:30.195696 (+ 563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:32:30.195701 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:32:30.195768 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:30.196686 (+ 918us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 09:32:30.196698 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:32:30.198254 (+ 1556us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 09:32:30.200373 (+ 2119us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:32:30.200392 (+ 19us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:32:30.200398 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:32:30.200647 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:32:30.201016 (+ 369us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:32:30.201019 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:32:30.201020 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:32:30.201108 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:32:30.201493 (+ 385us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 09:32:30.201499 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 09:32:30.201609 (+ 110us) client_negotiation.cc:769] Sending connection context
0809 09:32:30.201721 (+ 112us) client_negotiation.cc:240] Negotiation successful
0809 09:32:30.201849 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":267,"thread_start_us":106,"threads_started":1}
I20260809 09:32:30.206980 22664 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.20.125.190@KRBTEST.COM'} at 127.0.0.1:55612:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260809 09:32:30.207949 22664 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 09:32:30.213405 22875 tablet_service.cc:1467] Processing CreateTablet for tablet 78f5216f04dc41c38764ad34167af4e5 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=3878a7bece4e4b1282e25db1cc1d7eef]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 09:32:30.213490 23112 tablet_service.cc:1467] Processing CreateTablet for tablet 78f5216f04dc41c38764ad34167af4e5 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=3878a7bece4e4b1282e25db1cc1d7eef]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 09:32:30.213507 23305 tablet_service.cc:1467] Processing CreateTablet for tablet 78f5216f04dc41c38764ad34167af4e5 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=3878a7bece4e4b1282e25db1cc1d7eef]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 09:32:30.213822 22875 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 78f5216f04dc41c38764ad34167af4e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 09:32:30.213853 23305 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 78f5216f04dc41c38764ad34167af4e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 09:32:30.213889 23112 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 78f5216f04dc41c38764ad34167af4e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 09:32:30.218271 23470 tablet_bootstrap.cc:492] T 78f5216f04dc41c38764ad34167af4e5 P c405fafe9cca4eeea6ff3e56a6f8f61a: Bootstrap starting.
I20260809 09:32:30.219211 23471 tablet_bootstrap.cc:492] T 78f5216f04dc41c38764ad34167af4e5 P 41c5693686644259835b5ee95bca2fad: Bootstrap starting.
I20260809 09:32:30.219624 23470 tablet_bootstrap.cc:654] T 78f5216f04dc41c38764ad34167af4e5 P c405fafe9cca4eeea6ff3e56a6f8f61a: Neither blocks nor log segments found. Creating new log.
I20260809 09:32:30.220382 23471 tablet_bootstrap.cc:654] T 78f5216f04dc41c38764ad34167af4e5 P 41c5693686644259835b5ee95bca2fad: Neither blocks nor log segments found. Creating new log.
I20260809 09:32:30.220415 23469 tablet_bootstrap.cc:492] T 78f5216f04dc41c38764ad34167af4e5 P f326fb02673b498d958b36e7fd8eb682: Bootstrap starting.
I20260809 09:32:30.221365 23469 tablet_bootstrap.cc:654] T 78f5216f04dc41c38764ad34167af4e5 P f326fb02673b498d958b36e7fd8eb682: Neither blocks nor log segments found. Creating new log.
I20260809 09:32:30.221812 23471 tablet_bootstrap.cc:492] T 78f5216f04dc41c38764ad34167af4e5 P 41c5693686644259835b5ee95bca2fad: No bootstrap required, opened a new log
I20260809 09:32:30.221917 23471 ts_tablet_manager.cc:1397] T 78f5216f04dc41c38764ad34167af4e5 P 41c5693686644259835b5ee95bca2fad: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 09:32:30.222086 23470 tablet_bootstrap.cc:492] T 78f5216f04dc41c38764ad34167af4e5 P c405fafe9cca4eeea6ff3e56a6f8f61a: No bootstrap required, opened a new log
I20260809 09:32:30.222174 23470 ts_tablet_manager.cc:1397] T 78f5216f04dc41c38764ad34167af4e5 P c405fafe9cca4eeea6ff3e56a6f8f61a: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260809 09:32:30.222419 23471 raft_consensus.cc:348] T 78f5216f04dc41c38764ad34167af4e5 P 41c5693686644259835b5ee95bca2fad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c405fafe9cca4eeea6ff3e56a6f8f61a" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 36447 } } peers { permanent_uuid: "41c5693686644259835b5ee95bca2fad" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 46859 } } peers { permanent_uuid: "f326fb02673b498d958b36e7fd8eb682" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 42809 } }
I20260809 09:32:30.222611 23471 raft_consensus.cc:374] T 78f5216f04dc41c38764ad34167af4e5 P 41c5693686644259835b5ee95bca2fad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 09:32:30.222656 23471 raft_consensus.cc:729] T 78f5216f04dc41c38764ad34167af4e5 P 41c5693686644259835b5ee95bca2fad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41c5693686644259835b5ee95bca2fad, State: Initialized, Role: FOLLOWER
I20260809 09:32:30.222604 23470 raft_consensus.cc:348] T 78f5216f04dc41c38764ad34167af4e5 P c405fafe9cca4eeea6ff3e56a6f8f61a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c405fafe9cca4eeea6ff3e56a6f8f61a" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 36447 } } peers { permanent_uuid: "41c5693686644259835b5ee95bca2fad" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 46859 } } peers { permanent_uuid: "f326fb02673b498d958b36e7fd8eb682" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 42809 } }
I20260809 09:32:30.222805 23470 raft_consensus.cc:374] T 78f5216f04dc41c38764ad34167af4e5 P c405fafe9cca4eeea6ff3e56a6f8f61a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 09:32:30.222801 23471 consensus_queue.cc:260] T 78f5216f04dc41c38764ad34167af4e5 P 41c5693686644259835b5ee95bca2fad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c405fafe9cca4eeea6ff3e56a6f8f61a" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 36447 } } peers { permanent_uuid: "41c5693686644259835b5ee95bca2fad" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 46859 } } peers { permanent_uuid: "f326fb02673b498d958b36e7fd8eb682" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 42809 } }
I20260809 09:32:30.222870 23470 raft_consensus.cc:729] T 78f5216f04dc41c38764ad34167af4e5 P c405fafe9cca4eeea6ff3e56a6f8f61a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c405fafe9cca4eeea6ff3e56a6f8f61a, State: Initialized, Role: FOLLOWER
I20260809 09:32:30.222999 23470 consensus_queue.cc:260] T 78f5216f04dc41c38764ad34167af4e5 P c405fafe9cca4eeea6ff3e56a6f8f61a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c405fafe9cca4eeea6ff3e56a6f8f61a" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 36447 } } peers { permanent_uuid: "41c5693686644259835b5ee95bca2fad" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 46859 } } peers { permanent_uuid: "f326fb02673b498d958b36e7fd8eb682" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 42809 } }
I20260809 09:32:30.223090 23471 ts_tablet_manager.cc:1428] T 78f5216f04dc41c38764ad34167af4e5 P 41c5693686644259835b5ee95bca2fad: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260809 09:32:30.223155 23478 tablet_replica.cc:439] T 78f5216f04dc41c38764ad34167af4e5 P 41c5693686644259835b5ee95bca2fad: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c405fafe9cca4eeea6ff3e56a6f8f61a" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 36447 } } peers { permanent_uuid: "41c5693686644259835b5ee95bca2fad" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 46859 } } peers { permanent_uuid: "f326fb02673b498d958b36e7fd8eb682" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 42809 } } }
I20260809 09:32:30.223326 23478 tablet_replica.cc:442] T 78f5216f04dc41c38764ad34167af4e5 P 41c5693686644259835b5ee95bca2fad: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 09:32:30.223362 23470 ts_tablet_manager.cc:1428] T 78f5216f04dc41c38764ad34167af4e5 P c405fafe9cca4eeea6ff3e56a6f8f61a: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260809 09:32:30.223541 23477 tablet_replica.cc:439] T 78f5216f04dc41c38764ad34167af4e5 P c405fafe9cca4eeea6ff3e56a6f8f61a: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c405fafe9cca4eeea6ff3e56a6f8f61a" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 36447 } } peers { permanent_uuid: "41c5693686644259835b5ee95bca2fad" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 46859 } } peers { permanent_uuid: "f326fb02673b498d958b36e7fd8eb682" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 42809 } } }
I20260809 09:32:30.223660 23477 tablet_replica.cc:442] T 78f5216f04dc41c38764ad34167af4e5 P c405fafe9cca4eeea6ff3e56a6f8f61a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 09:32:30.224444 23469 tablet_bootstrap.cc:492] T 78f5216f04dc41c38764ad34167af4e5 P f326fb02673b498d958b36e7fd8eb682: No bootstrap required, opened a new log
I20260809 09:32:30.224560 23469 ts_tablet_manager.cc:1397] T 78f5216f04dc41c38764ad34167af4e5 P f326fb02673b498d958b36e7fd8eb682: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260809 09:32:30.225067 23469 raft_consensus.cc:348] T 78f5216f04dc41c38764ad34167af4e5 P f326fb02673b498d958b36e7fd8eb682 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c405fafe9cca4eeea6ff3e56a6f8f61a" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 36447 } } peers { permanent_uuid: "41c5693686644259835b5ee95bca2fad" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 46859 } } peers { permanent_uuid: "f326fb02673b498d958b36e7fd8eb682" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 42809 } }
I20260809 09:32:30.225198 23469 raft_consensus.cc:374] T 78f5216f04dc41c38764ad34167af4e5 P f326fb02673b498d958b36e7fd8eb682 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 09:32:30.225241 23469 raft_consensus.cc:729] T 78f5216f04dc41c38764ad34167af4e5 P f326fb02673b498d958b36e7fd8eb682 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f326fb02673b498d958b36e7fd8eb682, State: Initialized, Role: FOLLOWER
I20260809 09:32:30.225370 23469 consensus_queue.cc:260] T 78f5216f04dc41c38764ad34167af4e5 P f326fb02673b498d958b36e7fd8eb682 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c405fafe9cca4eeea6ff3e56a6f8f61a" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 36447 } } peers { permanent_uuid: "41c5693686644259835b5ee95bca2fad" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 46859 } } peers { permanent_uuid: "f326fb02673b498d958b36e7fd8eb682" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 42809 } }
I20260809 09:32:30.225678 23469 ts_tablet_manager.cc:1428] T 78f5216f04dc41c38764ad34167af4e5 P f326fb02673b498d958b36e7fd8eb682: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260809 09:32:30.225767 23480 tablet_replica.cc:439] T 78f5216f04dc41c38764ad34167af4e5 P f326fb02673b498d958b36e7fd8eb682: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c405fafe9cca4eeea6ff3e56a6f8f61a" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 36447 } } peers { permanent_uuid: "41c5693686644259835b5ee95bca2fad" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 46859 } } peers { permanent_uuid: "f326fb02673b498d958b36e7fd8eb682" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 42809 } } }
I20260809 09:32:30.225919 23480 tablet_replica.cc:442] T 78f5216f04dc41c38764ad34167af4e5 P f326fb02673b498d958b36e7fd8eb682: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 09:32:30.441833 23478 raft_consensus.cc:482] T 78f5216f04dc41c38764ad34167af4e5 P 41c5693686644259835b5ee95bca2fad [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 09:32:30.442028 23478 raft_consensus.cc:504] T 78f5216f04dc41c38764ad34167af4e5 P 41c5693686644259835b5ee95bca2fad [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c405fafe9cca4eeea6ff3e56a6f8f61a" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 36447 } } peers { permanent_uuid: "41c5693686644259835b5ee95bca2fad" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 46859 } } peers { permanent_uuid: "f326fb02673b498d958b36e7fd8eb682" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 42809 } }
I20260809 09:32:30.443272 23478 leader_election.cc:288] T 78f5216f04dc41c38764ad34167af4e5 P 41c5693686644259835b5ee95bca2fad [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c405fafe9cca4eeea6ff3e56a6f8f61a (127.20.125.129:36447), f326fb02673b498d958b36e7fd8eb682 (127.20.125.131:42809)
I20260809 09:32:30.448179 23462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:30.443680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.130:44015 (local address 127.20.125.129:36447)
0809 09:32:30.443841 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0809 09:32:30.443846 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 09:32:30.444014 (+ 168us) server_negotiation.cc:407] Connection header received
0809 09:32:30.444257 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:32:30.444263 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:32:30.444313 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:32:30.444355 (+ 42us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 09:32:30.444847 (+ 492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:30.445598 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:30.447183 (+ 1585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:30.447599 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:30.447744 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:30.447800 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0809 09:32:30.447992 (+ 192us) server_negotiation.cc:299] Negotiation successful
0809 09:32:30.448067 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 09:32:30.448199 23536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:30.443589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.129:36447 (local address 127.20.125.130:44015)
0809 09:32:30.443916 (+ 327us) negotiation.cc:107] Waiting for socket to connect
0809 09:32:30.443934 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 09:32:30.444106 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:32:30.444474 (+ 368us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:32:30.444499 (+ 25us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:32:30.444553 (+ 54us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 09:32:30.444723 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:32:30.444730 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:32:30.445758 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:32:30.445761 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:32:30.447056 (+ 1295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:32:30.447065 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:32:30.447747 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:32:30.447751 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:32:30.447859 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:30.447879 (+ 20us) client_negotiation.cc:769] Sending connection context
0809 09:32:30.447970 (+ 91us) client_negotiation.cc:240] Negotiation successful
0809 09:32:30.448041 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":233,"mutex_wait_us":2,"spinlock_wait_cycles":35968,"thread_start_us":125,"threads_started":1}
I20260809 09:32:30.448709 22906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78f5216f04dc41c38764ad34167af4e5" candidate_uuid: "41c5693686644259835b5ee95bca2fad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c405fafe9cca4eeea6ff3e56a6f8f61a" is_pre_election: true
I20260809 09:32:30.448871 22906 raft_consensus.cc:2454] T 78f5216f04dc41c38764ad34167af4e5 P c405fafe9cca4eeea6ff3e56a6f8f61a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41c5693686644259835b5ee95bca2fad in term 0.
I20260809 09:32:30.449224 23036 leader_election.cc:302] T 78f5216f04dc41c38764ad34167af4e5 P 41c5693686644259835b5ee95bca2fad [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41c5693686644259835b5ee95bca2fad, c405fafe9cca4eeea6ff3e56a6f8f61a; no voters:
I20260809 09:32:30.449488 23478 raft_consensus.cc:2781] T 78f5216f04dc41c38764ad34167af4e5 P 41c5693686644259835b5ee95bca2fad [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 09:32:30.449559 23478 raft_consensus.cc:482] T 78f5216f04dc41c38764ad34167af4e5 P 41c5693686644259835b5ee95bca2fad [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 09:32:30.449595 23478 raft_consensus.cc:3037] T 78f5216f04dc41c38764ad34167af4e5 P 41c5693686644259835b5ee95bca2fad [term 0 FOLLOWER]: Advancing to term 1
I20260809 09:32:30.450007 23537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:30.444034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.131:42809 (local address 127.20.125.130:36525)
0809 09:32:30.444400 (+ 366us) negotiation.cc:107] Waiting for socket to connect
0809 09:32:30.444414 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 09:32:30.444561 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:32:30.445743 (+ 1182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:32:30.445746 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:32:30.445764 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 09:32:30.445926 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:32:30.445932 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:32:30.446985 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:32:30.446988 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:32:30.448591 (+ 1603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:32:30.448602 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:32:30.449538 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:32:30.449540 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:32:30.449674 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:30.449696 (+ 22us) client_negotiation.cc:769] Sending connection context
0809 09:32:30.449796 (+ 100us) client_negotiation.cc:240] Negotiation successful
0809 09:32:30.449861 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":305,"mutex_wait_us":34,"spinlock_wait_cycles":4736,"thread_start_us":82,"threads_started":1}
I20260809 09:32:30.450009 23459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:30.444129 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.130:36525 (local address 127.20.125.131:42809)
0809 09:32:30.444239 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0809 09:32:30.444243 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 09:32:30.444529 (+ 286us) server_negotiation.cc:407] Connection header received
0809 09:32:30.444672 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:32:30.444676 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:32:30.444729 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:32:30.444812 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 09:32:30.446072 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:30.446864 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:30.448706 (+ 1842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:30.449288 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:30.449410 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:30.449469 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0809 09:32:30.449815 (+ 346us) server_negotiation.cc:299] Negotiation successful
0809 09:32:30.449900 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260809 09:32:30.450824 23478 raft_consensus.cc:504] T 78f5216f04dc41c38764ad34167af4e5 P 41c5693686644259835b5ee95bca2fad [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c405fafe9cca4eeea6ff3e56a6f8f61a" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 36447 } } peers { permanent_uuid: "41c5693686644259835b5ee95bca2fad" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 46859 } } peers { permanent_uuid: "f326fb02673b498d958b36e7fd8eb682" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 42809 } }
I20260809 09:32:30.450920 23329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78f5216f04dc41c38764ad34167af4e5" candidate_uuid: "41c5693686644259835b5ee95bca2fad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f326fb02673b498d958b36e7fd8eb682" is_pre_election: true
I20260809 09:32:30.451115 23329 raft_consensus.cc:2454] T 78f5216f04dc41c38764ad34167af4e5 P f326fb02673b498d958b36e7fd8eb682 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41c5693686644259835b5ee95bca2fad in term 0.
I20260809 09:32:30.451195 23478 leader_election.cc:288] T 78f5216f04dc41c38764ad34167af4e5 P 41c5693686644259835b5ee95bca2fad [CANDIDATE]: Term 1 election: Requested vote from peers c405fafe9cca4eeea6ff3e56a6f8f61a (127.20.125.129:36447), f326fb02673b498d958b36e7fd8eb682 (127.20.125.131:42809)
I20260809 09:32:30.451606 23329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78f5216f04dc41c38764ad34167af4e5" candidate_uuid: "41c5693686644259835b5ee95bca2fad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f326fb02673b498d958b36e7fd8eb682"
I20260809 09:32:30.451603 22906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78f5216f04dc41c38764ad34167af4e5" candidate_uuid: "41c5693686644259835b5ee95bca2fad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c405fafe9cca4eeea6ff3e56a6f8f61a"
I20260809 09:32:30.451743 23329 raft_consensus.cc:3037] T 78f5216f04dc41c38764ad34167af4e5 P f326fb02673b498d958b36e7fd8eb682 [term 0 FOLLOWER]: Advancing to term 1
I20260809 09:32:30.451767 22906 raft_consensus.cc:3037] T 78f5216f04dc41c38764ad34167af4e5 P c405fafe9cca4eeea6ff3e56a6f8f61a [term 0 FOLLOWER]: Advancing to term 1
I20260809 09:32:30.452992 22906 raft_consensus.cc:2454] T 78f5216f04dc41c38764ad34167af4e5 P c405fafe9cca4eeea6ff3e56a6f8f61a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41c5693686644259835b5ee95bca2fad in term 1.
I20260809 09:32:30.452992 23329 raft_consensus.cc:2454] T 78f5216f04dc41c38764ad34167af4e5 P f326fb02673b498d958b36e7fd8eb682 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41c5693686644259835b5ee95bca2fad in term 1.
I20260809 09:32:30.453284 23036 leader_election.cc:302] T 78f5216f04dc41c38764ad34167af4e5 P 41c5693686644259835b5ee95bca2fad [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41c5693686644259835b5ee95bca2fad, c405fafe9cca4eeea6ff3e56a6f8f61a; no voters:
I20260809 09:32:30.453476 23478 raft_consensus.cc:2781] T 78f5216f04dc41c38764ad34167af4e5 P 41c5693686644259835b5ee95bca2fad [term 1 FOLLOWER]: Leader election won for term 1
I20260809 09:32:30.453681 23478 raft_consensus.cc:686] T 78f5216f04dc41c38764ad34167af4e5 P 41c5693686644259835b5ee95bca2fad [term 1 LEADER]: Becoming Leader. State: Replica: 41c5693686644259835b5ee95bca2fad, State: Running, Role: LEADER
I20260809 09:32:30.453918 23478 consensus_queue.cc:237] T 78f5216f04dc41c38764ad34167af4e5 P 41c5693686644259835b5ee95bca2fad [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c405fafe9cca4eeea6ff3e56a6f8f61a" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 36447 } } peers { permanent_uuid: "41c5693686644259835b5ee95bca2fad" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 46859 } } peers { permanent_uuid: "f326fb02673b498d958b36e7fd8eb682" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 42809 } }
I20260809 09:32:30.455350 23538 tablet_replica.cc:439] T 78f5216f04dc41c38764ad34167af4e5 P 41c5693686644259835b5ee95bca2fad: TxnStatusTablet state changed. Reason: New leader 41c5693686644259835b5ee95bca2fad. Latest consensus state: current_term: 1 leader_uuid: "41c5693686644259835b5ee95bca2fad" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c405fafe9cca4eeea6ff3e56a6f8f61a" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 36447 } } peers { permanent_uuid: "41c5693686644259835b5ee95bca2fad" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 46859 } } peers { permanent_uuid: "f326fb02673b498d958b36e7fd8eb682" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 42809 } } }
I20260809 09:32:30.455531 23538 tablet_replica.cc:442] T 78f5216f04dc41c38764ad34167af4e5 P 41c5693686644259835b5ee95bca2fad: This TxnStatusTablet replica's current role is: LEADER
I20260809 09:32:30.455992 23541 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260809 09:32:30.456184 23541 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260809 09:32:30.456652 22666 catalog_manager.cc:5495] T 78f5216f04dc41c38764ad34167af4e5 P 41c5693686644259835b5ee95bca2fad reported cstate change: term changed from 0 to 1, leader changed from <none> to 41c5693686644259835b5ee95bca2fad (127.20.125.130). New cstate: current_term: 1 leader_uuid: "41c5693686644259835b5ee95bca2fad" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c405fafe9cca4eeea6ff3e56a6f8f61a" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 36447 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "41c5693686644259835b5ee95bca2fad" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 46859 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f326fb02673b498d958b36e7fd8eb682" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 42809 } health_report { overall_health: UNKNOWN } } }
I20260809 09:32:30.470650 23468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:30.465953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33390 (local address 127.20.125.130:46859)
0809 09:32:30.466080 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0809 09:32:30.466083 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 09:32:30.466256 (+ 173us) server_negotiation.cc:407] Connection header received
0809 09:32:30.466388 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:32:30.466390 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:32:30.466441 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:32:30.466587 (+ 146us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 09:32:30.466887 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:30.467420 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:30.468685 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:30.468911 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:30.469002 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:30.469492 (+ 490us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 09:32:30.470173 (+ 681us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 09:32:30.470278 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0809 09:32:30.470419 (+ 141us) server_negotiation.cc:299] Negotiation successful
0809 09:32:30.470531 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260809 09:32:30.470667 23530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:30.465848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.130:46859 (local address 127.0.0.1:33390)
0809 09:32:30.466099 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0809 09:32:30.466114 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 09:32:30.466244 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:32:30.466616 (+ 372us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:32:30.466620 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:32:30.466629 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 09:32:30.466756 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:32:30.466762 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:32:30.467539 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:32:30.467543 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:32:30.468539 (+ 996us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:32:30.468551 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:32:30.469038 (+ 487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:32:30.469042 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:32:30.469141 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:30.469205 (+ 64us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 09:32:30.470290 (+ 1085us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 09:32:30.470308 (+ 18us) client_negotiation.cc:769] Sending connection context
0809 09:32:30.470392 (+ 84us) client_negotiation.cc:240] Negotiation successful
0809 09:32:30.470510 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149}
I20260809 09:32:30.474103 22906 raft_consensus.cc:1261] T 78f5216f04dc41c38764ad34167af4e5 P c405fafe9cca4eeea6ff3e56a6f8f61a [term 1 FOLLOWER]: Refusing update from remote peer 41c5693686644259835b5ee95bca2fad: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 09:32:30.474406 23329 raft_consensus.cc:1261] T 78f5216f04dc41c38764ad34167af4e5 P f326fb02673b498d958b36e7fd8eb682 [term 1 FOLLOWER]: Refusing update from remote peer 41c5693686644259835b5ee95bca2fad: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 09:32:30.474713 23478 consensus_queue.cc:1035] T 78f5216f04dc41c38764ad34167af4e5 P 41c5693686644259835b5ee95bca2fad [LEADER]: Connected to new peer: Peer: permanent_uuid: "c405fafe9cca4eeea6ff3e56a6f8f61a" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 36447 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 09:32:30.474939 23538 consensus_queue.cc:1035] T 78f5216f04dc41c38764ad34167af4e5 P 41c5693686644259835b5ee95bca2fad [LEADER]: Connected to new peer: Peer: permanent_uuid: "f326fb02673b498d958b36e7fd8eb682" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 42809 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 09:32:30.476634 23480 tablet_replica.cc:439] T 78f5216f04dc41c38764ad34167af4e5 P f326fb02673b498d958b36e7fd8eb682: TxnStatusTablet state changed. Reason: New leader 41c5693686644259835b5ee95bca2fad. Latest consensus state: current_term: 1 leader_uuid: "41c5693686644259835b5ee95bca2fad" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c405fafe9cca4eeea6ff3e56a6f8f61a" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 36447 } } peers { permanent_uuid: "41c5693686644259835b5ee95bca2fad" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 46859 } } peers { permanent_uuid: "f326fb02673b498d958b36e7fd8eb682" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 42809 } } }
I20260809 09:32:30.476670 23477 tablet_replica.cc:439] T 78f5216f04dc41c38764ad34167af4e5 P c405fafe9cca4eeea6ff3e56a6f8f61a: TxnStatusTablet state changed. Reason: New leader 41c5693686644259835b5ee95bca2fad. Latest consensus state: current_term: 1 leader_uuid: "41c5693686644259835b5ee95bca2fad" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c405fafe9cca4eeea6ff3e56a6f8f61a" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 36447 } } peers { permanent_uuid: "41c5693686644259835b5ee95bca2fad" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 46859 } } peers { permanent_uuid: "f326fb02673b498d958b36e7fd8eb682" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 42809 } } }
I20260809 09:32:30.476773 23480 tablet_replica.cc:442] T 78f5216f04dc41c38764ad34167af4e5 P f326fb02673b498d958b36e7fd8eb682: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 09:32:30.476783 23477 tablet_replica.cc:442] T 78f5216f04dc41c38764ad34167af4e5 P c405fafe9cca4eeea6ff3e56a6f8f61a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 09:32:30.478960 23478 tablet_replica.cc:439] T 78f5216f04dc41c38764ad34167af4e5 P 41c5693686644259835b5ee95bca2fad: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "41c5693686644259835b5ee95bca2fad" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c405fafe9cca4eeea6ff3e56a6f8f61a" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 36447 } } peers { permanent_uuid: "41c5693686644259835b5ee95bca2fad" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 46859 } } peers { permanent_uuid: "f326fb02673b498d958b36e7fd8eb682" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 42809 } } }
I20260809 09:32:30.479070 23478 tablet_replica.cc:442] T 78f5216f04dc41c38764ad34167af4e5 P 41c5693686644259835b5ee95bca2fad: This TxnStatusTablet replica's current role is: LEADER
I20260809 09:32:30.479216 23478 tablet_replica.cc:439] T 78f5216f04dc41c38764ad34167af4e5 P 41c5693686644259835b5ee95bca2fad: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "41c5693686644259835b5ee95bca2fad" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c405fafe9cca4eeea6ff3e56a6f8f61a" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 36447 } } peers { permanent_uuid: "41c5693686644259835b5ee95bca2fad" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 46859 } } peers { permanent_uuid: "f326fb02673b498d958b36e7fd8eb682" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 42809 } } }
I20260809 09:32:30.479310 23478 tablet_replica.cc:442] T 78f5216f04dc41c38764ad34167af4e5 P 41c5693686644259835b5ee95bca2fad: This TxnStatusTablet replica's current role is: LEADER
I20260809 09:32:30.479406 23493 mvcc.cc:205] Tried to move back new op lower bound from 7316553525136470016 to 7316553525061840896. Current Snapshot: MvccSnapshot[applied={T|T < 7316553525136470016 or (T in {7316553525136470016})}]
I20260809 09:32:30.479889 23477 tablet_replica.cc:439] T 78f5216f04dc41c38764ad34167af4e5 P c405fafe9cca4eeea6ff3e56a6f8f61a: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "41c5693686644259835b5ee95bca2fad" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c405fafe9cca4eeea6ff3e56a6f8f61a" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 36447 } } peers { permanent_uuid: "41c5693686644259835b5ee95bca2fad" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 46859 } } peers { permanent_uuid: "f326fb02673b498d958b36e7fd8eb682" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 42809 } } }
I20260809 09:32:30.479993 23477 tablet_replica.cc:442] T 78f5216f04dc41c38764ad34167af4e5 P c405fafe9cca4eeea6ff3e56a6f8f61a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 09:32:30.481642 23480 tablet_replica.cc:439] T 78f5216f04dc41c38764ad34167af4e5 P f326fb02673b498d958b36e7fd8eb682: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "41c5693686644259835b5ee95bca2fad" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c405fafe9cca4eeea6ff3e56a6f8f61a" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 36447 } } peers { permanent_uuid: "41c5693686644259835b5ee95bca2fad" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 46859 } } peers { permanent_uuid: "f326fb02673b498d958b36e7fd8eb682" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 42809 } } }
I20260809 09:32:30.481801 23480 tablet_replica.cc:442] T 78f5216f04dc41c38764ad34167af4e5 P f326fb02673b498d958b36e7fd8eb682: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 09:32:30.481920 23487 mvcc.cc:205] Tried to move back new op lower bound from 7316553525136470016 to 7316553525061840896. Current Snapshot: MvccSnapshot[applied={T|T < 7316553525136470016 or (T in {7316553525136470016})}]
I20260809 09:32:30.488961 23462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:30.484683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38672 (local address 127.20.125.129:36447)
0809 09:32:30.484806 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0809 09:32:30.484811 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 09:32:30.484827 (+ 16us) server_negotiation.cc:407] Connection header received
0809 09:32:30.484965 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:32:30.484968 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:32:30.485032 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:32:30.485144 (+ 112us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 09:32:30.485411 (+ 267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:30.486227 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:30.487173 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:30.487543 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:30.487652 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:30.487969 (+ 317us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 09:32:30.488534 (+ 565us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 09:32:30.488683 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0809 09:32:30.488762 (+ 79us) server_negotiation.cc:299] Negotiation successful
0809 09:32:30.488824 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260809 09:32:30.502206 23468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:30.497848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.129:51701 (local address 127.20.125.130:46859)
0809 09:32:30.497992 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0809 09:32:30.497996 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 09:32:30.498314 (+ 318us) server_negotiation.cc:407] Connection header received
0809 09:32:30.498439 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:32:30.498443 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:32:30.498519 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:32:30.498604 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 09:32:30.499198 (+ 594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:30.500006 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:30.500911 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:30.501268 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:30.501372 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:30.501675 (+ 303us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 09:32:30.501762 (+ 87us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 09:32:30.501837 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0809 09:32:30.501996 (+ 159us) server_negotiation.cc:299] Negotiation successful
0809 09:32:30.502054 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260809 09:32:30.502211 23559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:30.497739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.130:46859 (local address 127.20.125.129:51701)
0809 09:32:30.498071 (+ 332us) negotiation.cc:107] Waiting for socket to connect
0809 09:32:30.498084 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 09:32:30.498332 (+ 248us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:32:30.498616 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:32:30.498618 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:32:30.498625 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 09:32:30.499074 (+ 449us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:32:30.499082 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:32:30.500146 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:32:30.500149 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:32:30.500788 (+ 639us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:32:30.500797 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:32:30.501404 (+ 607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:32:30.501407 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:32:30.501484 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:30.501534 (+ 50us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 09:32:30.501882 (+ 348us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 09:32:30.501890 (+ 8us) client_negotiation.cc:769] Sending connection context
0809 09:32:30.501967 (+ 77us) client_negotiation.cc:240] Negotiation successful
0809 09:32:30.502034 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":122,"threads_started":1}
I20260809 09:32:30.510520 23329 raft_consensus.cc:1261] T 6e8ecb32d2e94f7cb93eb8fc7b6e5f4e P f326fb02673b498d958b36e7fd8eb682 [term 1 FOLLOWER]: Refusing update from remote peer c405fafe9cca4eeea6ff3e56a6f8f61a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 09:32:30.510509 23141 raft_consensus.cc:1261] T 6e8ecb32d2e94f7cb93eb8fc7b6e5f4e P 41c5693686644259835b5ee95bca2fad [term 1 FOLLOWER]: Refusing update from remote peer c405fafe9cca4eeea6ff3e56a6f8f61a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 09:32:30.511003 23517 consensus_queue.cc:1035] T 6e8ecb32d2e94f7cb93eb8fc7b6e5f4e P c405fafe9cca4eeea6ff3e56a6f8f61a [LEADER]: Connected to new peer: Peer: permanent_uuid: "f326fb02673b498d958b36e7fd8eb682" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 42809 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 09:32:30.511189 23477 consensus_queue.cc:1035] T 6e8ecb32d2e94f7cb93eb8fc7b6e5f4e P c405fafe9cca4eeea6ff3e56a6f8f61a [LEADER]: Connected to new peer: Peer: permanent_uuid: "41c5693686644259835b5ee95bca2fad" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 46859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 09:32:30.536782 23567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:30.532441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.129:36447 (local address 127.20.125.130:52891)
0809 09:32:30.532801 (+ 360us) negotiation.cc:107] Waiting for socket to connect
0809 09:32:30.532861 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0809 09:32:30.533121 (+ 260us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:32:30.533356 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:32:30.533359 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:32:30.533366 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 09:32:30.533526 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:32:30.533532 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:32:30.534392 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:32:30.534395 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:32:30.535297 (+ 902us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:32:30.535309 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:32:30.535973 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:32:30.535976 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:32:30.536086 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:30.536138 (+ 52us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 09:32:30.536455 (+ 317us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 09:32:30.536464 (+ 9us) client_negotiation.cc:769] Sending connection context
0809 09:32:30.536531 (+ 67us) client_negotiation.cc:240] Negotiation successful
0809 09:32:30.536611 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":232,"thread_start_us":126,"threads_started":1}
I20260809 09:32:30.536761 23462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:30.532474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.130:52891 (local address 127.20.125.129:36447)
0809 09:32:30.532609 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0809 09:32:30.532614 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 09:32:30.533110 (+ 496us) server_negotiation.cc:407] Connection header received
0809 09:32:30.533206 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:32:30.533209 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:32:30.533252 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:32:30.533325 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 09:32:30.533639 (+ 314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:30.534242 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:30.535447 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:30.535760 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:30.535826 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:30.536284 (+ 458us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 09:32:30.536365 (+ 81us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 09:32:30.536450 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0809 09:32:30.536554 (+ 104us) server_negotiation.cc:299] Negotiation successful
0809 09:32:30.536611 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260809 09:32:30.562209 22666 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55604:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 09:32:30.562399 22666 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55604:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 09:32:30.564716 22666 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0ef673811e7443ac84e6e78369057d13: Sending DeleteTablet for 3 replicas of tablet 6e8ecb32d2e94f7cb93eb8fc7b6e5f4e
I20260809 09:32:30.565305 22875 tablet_service.cc:1514] Processing DeleteTablet for tablet 6e8ecb32d2e94f7cb93eb8fc7b6e5f4e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 09:32:30 UTC) from {username='kudu', principal='kudu/127.20.125.190@KRBTEST.COM'} at 127.0.0.1:38664
I20260809 09:32:30.565658 23572 tablet_replica.cc:330] T 6e8ecb32d2e94f7cb93eb8fc7b6e5f4e P c405fafe9cca4eeea6ff3e56a6f8f61a: stopping tablet replica
I20260809 09:32:30.565804 23572 raft_consensus.cc:2229] T 6e8ecb32d2e94f7cb93eb8fc7b6e5f4e P c405fafe9cca4eeea6ff3e56a6f8f61a [term 1 LEADER]: Raft consensus shutting down.
I20260809 09:32:30.566056 23112 tablet_service.cc:1514] Processing DeleteTablet for tablet 6e8ecb32d2e94f7cb93eb8fc7b6e5f4e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 09:32:30 UTC) from {username='kudu', principal='kudu/127.20.125.190@KRBTEST.COM'} at 127.0.0.1:33388
I20260809 09:32:30.566126 23572 raft_consensus.cc:2258] T 6e8ecb32d2e94f7cb93eb8fc7b6e5f4e P c405fafe9cca4eeea6ff3e56a6f8f61a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 09:32:30.566809 23305 tablet_service.cc:1514] Processing DeleteTablet for tablet 6e8ecb32d2e94f7cb93eb8fc7b6e5f4e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 09:32:30 UTC) from {username='kudu', principal='kudu/127.20.125.190@KRBTEST.COM'} at 127.0.0.1:47000
I20260809 09:32:30.566900 23572 ts_tablet_manager.cc:1905] T 6e8ecb32d2e94f7cb93eb8fc7b6e5f4e P c405fafe9cca4eeea6ff3e56a6f8f61a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 09:32:30.568462 23573 tablet_replica.cc:330] T 6e8ecb32d2e94f7cb93eb8fc7b6e5f4e P 41c5693686644259835b5ee95bca2fad: stopping tablet replica
I20260809 09:32:30.568645 23573 raft_consensus.cc:2229] T 6e8ecb32d2e94f7cb93eb8fc7b6e5f4e P 41c5693686644259835b5ee95bca2fad [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 09:32:30.568806 23573 raft_consensus.cc:2258] T 6e8ecb32d2e94f7cb93eb8fc7b6e5f4e P 41c5693686644259835b5ee95bca2fad [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 09:32:30.569576 23573 ts_tablet_manager.cc:1905] T 6e8ecb32d2e94f7cb93eb8fc7b6e5f4e P 41c5693686644259835b5ee95bca2fad: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 09:32:30.570181 23572 ts_tablet_manager.cc:1918] T 6e8ecb32d2e94f7cb93eb8fc7b6e5f4e P c405fafe9cca4eeea6ff3e56a6f8f61a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 09:32:30.570274 23572 log.cc:1196] T 6e8ecb32d2e94f7cb93eb8fc7b6e5f4e P c405fafe9cca4eeea6ff3e56a6f8f61a: Deleting WAL directory at /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786267941944493-20982-0/minicluster-data/ts-0/wal/wals/6e8ecb32d2e94f7cb93eb8fc7b6e5f4e
I20260809 09:32:30.570611 23572 ts_tablet_manager.cc:1939] T 6e8ecb32d2e94f7cb93eb8fc7b6e5f4e P c405fafe9cca4eeea6ff3e56a6f8f61a: Deleting consensus metadata
I20260809 09:32:30.571338 22639 catalog_manager.cc:4841] TS c405fafe9cca4eeea6ff3e56a6f8f61a (127.20.125.129:36447): tablet 6e8ecb32d2e94f7cb93eb8fc7b6e5f4e (table test-table [id=aeab29f88d714c90ac05ecb005b5b550]) successfully deleted
I20260809 09:32:30.573814 23573 ts_tablet_manager.cc:1918] T 6e8ecb32d2e94f7cb93eb8fc7b6e5f4e P 41c5693686644259835b5ee95bca2fad: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 09:32:30.574370 23573 log.cc:1196] T 6e8ecb32d2e94f7cb93eb8fc7b6e5f4e P 41c5693686644259835b5ee95bca2fad: Deleting WAL directory at /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786267941944493-20982-0/minicluster-data/ts-1/wal/wals/6e8ecb32d2e94f7cb93eb8fc7b6e5f4e
I20260809 09:32:30.574790 23573 ts_tablet_manager.cc:1939] T 6e8ecb32d2e94f7cb93eb8fc7b6e5f4e P 41c5693686644259835b5ee95bca2fad: Deleting consensus metadata
I20260809 09:32:30.575610 22639 catalog_manager.cc:4841] TS 41c5693686644259835b5ee95bca2fad (127.20.125.130:46859): tablet 6e8ecb32d2e94f7cb93eb8fc7b6e5f4e (table test-table [id=aeab29f88d714c90ac05ecb005b5b550]) successfully deleted
I20260809 09:32:30.576215 23575 tablet_replica.cc:330] T 6e8ecb32d2e94f7cb93eb8fc7b6e5f4e P f326fb02673b498d958b36e7fd8eb682: stopping tablet replica
I20260809 09:32:30.576372 23575 raft_consensus.cc:2229] T 6e8ecb32d2e94f7cb93eb8fc7b6e5f4e P f326fb02673b498d958b36e7fd8eb682 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 09:32:30.576514 23575 raft_consensus.cc:2258] T 6e8ecb32d2e94f7cb93eb8fc7b6e5f4e P f326fb02673b498d958b36e7fd8eb682 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 09:32:30.577852 23575 ts_tablet_manager.cc:1905] T 6e8ecb32d2e94f7cb93eb8fc7b6e5f4e P f326fb02673b498d958b36e7fd8eb682: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 09:32:30.581435 23575 ts_tablet_manager.cc:1918] T 6e8ecb32d2e94f7cb93eb8fc7b6e5f4e P f326fb02673b498d958b36e7fd8eb682: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
Aug 09 09:32:30 dist-test-slave-7kzw krb5kdc[22595](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786267949, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.20.125.129@KRBTEST.COM
I20260809 09:32:30.581547 23575 log.cc:1196] T 6e8ecb32d2e94f7cb93eb8fc7b6e5f4e P f326fb02673b498d958b36e7fd8eb682: Deleting WAL directory at /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786267941944493-20982-0/minicluster-data/ts-2/wal/wals/6e8ecb32d2e94f7cb93eb8fc7b6e5f4e
I20260809 09:32:30.581838 23575 ts_tablet_manager.cc:1939] T 6e8ecb32d2e94f7cb93eb8fc7b6e5f4e P f326fb02673b498d958b36e7fd8eb682: Deleting consensus metadata
I20260809 09:32:30.582695 22639 catalog_manager.cc:4841] TS f326fb02673b498d958b36e7fd8eb682 (127.20.125.131:42809): tablet 6e8ecb32d2e94f7cb93eb8fc7b6e5f4e (table test-table [id=aeab29f88d714c90ac05ecb005b5b550]) successfully deleted
I20260809 09:32:30.586990 23462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:30.570020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38678 (local address 127.20.125.129:36447)
0809 09:32:30.570127 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0809 09:32:30.570131 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 09:32:30.572109 (+ 1978us) server_negotiation.cc:407] Connection header received
0809 09:32:30.572319 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:32:30.572322 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:32:30.572369 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:32:30.572419 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:32:30.573514 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:30.574719 (+ 1205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:30.575980 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:30.576340 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:30.576443 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:30.582415 (+ 5972us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:32:30.582428 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:32:30.582443 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:32:30.582510 (+ 67us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:32:30.584888 (+ 2378us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:32:30.585433 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:32:30.585441 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:32:30.585442 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:32:30.585556 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:32:30.585878 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:32:30.585882 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:32:30.585884 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:32:30.586193 (+ 309us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:32:30.586402 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0809 09:32:30.586676 (+ 274us) server_negotiation.cc:299] Negotiation successful
0809 09:32:30.586806 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
W20260809 09:32:30.587623 22975 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38678
I20260809 09:32:30.599753 22740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:30.589616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55618 (local address 127.20.125.190:44501)
0809 09:32:30.589740 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0809 09:32:30.589746 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 09:32:30.590761 (+ 1015us) server_negotiation.cc:407] Connection header received
0809 09:32:30.590876 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:32:30.590880 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:32:30.590937 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:32:30.591054 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:32:30.591863 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:30.592763 (+ 900us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:30.593900 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:30.594279 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:30.594391 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:30.596074 (+ 1683us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:32:30.596078 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:32:30.596080 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:32:30.596108 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:32:30.597919 (+ 1811us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:32:30.598402 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:32:30.598406 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:32:30.598407 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:32:30.598526 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:32:30.598836 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:32:30.598839 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:32:30.598841 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:32:30.599077 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:32:30.599227 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0809 09:32:30.599466 (+ 239us) server_negotiation.cc:299] Negotiation successful
0809 09:32:30.599590 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
W20260809 09:32:30.600431 22666 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55618
I20260809 09:32:30.601913 20982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu with pid 22772
I20260809 09:32:30.628628 20982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu with pid 23001
I20260809 09:32:30.641676 20982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu with pid 23227
I20260809 09:32:30.653323 20982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu with pid 22617
2026-08-09T09:32:30Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (3354 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786267941944493-20982-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 09:32:30 dist-test-slave-7kzw krb5kdc[23584](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 09:32:30 dist-test-slave-7kzw krb5kdc[23584](info): set up 2 sockets
Aug 09 09:32:30 dist-test-slave-7kzw krb5kdc[23584](info): commencing operation
krb5kdc: starting...
W20260809 09:32:31.910533 20982 mini_kdc.cc:121] Time spent starting KDC: real 1.228s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786267941944493-20982-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786267941944493-20982-0/krb5kdc/test-user.keytab.
Aug 09 09:32:32 dist-test-slave-7kzw krb5kdc[23584](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786267952, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T09:32:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T09:32:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.125.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.125.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.125.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.190.keytab.
Entry for principal kudu/127.20.125.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.190.keytab.
I20260809 09:32:32.047641 20982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
/tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786267941944493-20982-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786267941944493-20982-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786267941944493-20982-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786267941944493-20982-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.125.190:41195
--webserver_interface=127.20.125.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.190.keytab
--principal=kudu/127.20.125.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.125.148:33107
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.125.190:41195
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786267941944493-20982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786267941944493-20982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786267941944493-20982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 09:32:32.162999 23600 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 09:32:32.163257 23600 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 09:32:32.163298 23600 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 09:32:32.167299 23600 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 09:32:32.167353 23600 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260809 09:32:32.167371 23600 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 09:32:32.167388 23600 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 09:32:32.167404 23600 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 09:32:32.167420 23600 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 09:32:32.172633 23600 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.125.148:33107
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786267941944493-20982-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786267941944493-20982-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.125.190:41195
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.190.keytab
--principal=kudu/127.20.125.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.125.190:41195
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786267941944493-20982-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.125.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23600
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786267941944493-20982-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 08:38:55 UTC on ccycloud.quasar-njjleu.root.comops.site
build id 4575935
I20260809 09:32:32.173748 23600 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 09:32:32.174865 23600 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 09:32:32.183892 23610 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:32:32.183897 23609 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:32:32.183897 23600 server_base.cc:1034] running on GCE node
W20260809 09:32:32.183916 23614 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:32:32.184719 23600 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 09:32:32.185823 23600 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 09:32:32.187019 23600 hybrid_clock.cc:648] HybridClock initialized: now 1786267952186996 us; error 34 us; skew 500 ppm
Aug 09 09:32:32 dist-test-slave-7kzw krb5kdc[23584](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786267952, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 09:32:32.190008 23600 init.cc:377] Logged in from keytab as kudu/127.20.125.190@KRBTEST.COM (short username kudu)
I20260809 09:32:32.190917 23600 webserver.cc:460] Webserver started at http://127.20.125.190:35569/ using document root <none> and password file <none>
I20260809 09:32:32.191452 23600 fs_manager.cc:359] Metadata directory not provided
I20260809 09:32:32.191514 23600 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 09:32:32.191677 23600 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 09:32:32.193660 23600 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786267941944493-20982-0/minicluster-data/master-0/data/instance:
uuid: "22534bf98c554d6c8acdd00f8d2a9a29"
format_stamp: "Formatted at 2026-08-09 09:32:32 on dist-test-slave-7kzw"
server_key: "996eec21781b1a0c02bca3210588fa4b"
server_key_iv: "bd5f424ac15627322ad21ff8abdbfbc7"
server_key_version: "encryptionkey@0"
I20260809 09:32:32.194222 23600 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786267941944493-20982-0/minicluster-data/master-0/wal/instance:
uuid: "22534bf98c554d6c8acdd00f8d2a9a29"
format_stamp: "Formatted at 2026-08-09 09:32:32 on dist-test-slave-7kzw"
server_key: "996eec21781b1a0c02bca3210588fa4b"
server_key_iv: "bd5f424ac15627322ad21ff8abdbfbc7"
server_key_version: "encryptionkey@0"
I20260809 09:32:32.197980 23600 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260809 09:32:32.200525 23622 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 09:32:32.201874 23600 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 09:32:32.202039 23600 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786267941944493-20982-0/minicluster-data/master-0/data,/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786267941944493-20982-0/minicluster-data/master-0/wal
uuid: "22534bf98c554d6c8acdd00f8d2a9a29"
format_stamp: "Formatted at 2026-08-09 09:32:32 on dist-test-slave-7kzw"
server_key: "996eec21781b1a0c02bca3210588fa4b"
server_key_iv: "bd5f424ac15627322ad21ff8abdbfbc7"
server_key_version: "encryptionkey@0"
I20260809 09:32:32.202152 23600 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786267941944493-20982-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786267941944493-20982-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786267941944493-20982-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 09:32:32.222970 23600 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 09:32:32.226104 23600 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 09:32:32.226311 23600 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 09:32:32.235046 23600 rpc_server.cc:307] RPC server started. Bound to: 127.20.125.190:41195
I20260809 09:32:32.235183 23715 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.125.190:41195 every 8 connection(s)
I20260809 09:32:32.236048 23600 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786267941944493-20982-0/minicluster-data/master-0/data/info.pb
I20260809 09:32:32.239110 23718 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 09:32:32.245379 20982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu as pid 23600
I20260809 09:32:32.245514 20982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786267941944493-20982-0/minicluster-data/master-0/wal/instance
I20260809 09:32:32.245486 23718 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 22534bf98c554d6c8acdd00f8d2a9a29: Bootstrap starting.
I20260809 09:32:32.245867 20982 external_mini_cluster.cc:1428] Setting key b344c60b523130262896890b2fa2d061
I20260809 09:32:32.248526 23718 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 22534bf98c554d6c8acdd00f8d2a9a29: Neither blocks nor log segments found. Creating new log.
I20260809 09:32:32.249344 23718 log.cc:824] T 00000000000000000000000000000000 P 22534bf98c554d6c8acdd00f8d2a9a29: Log is configured to *not* fsync() on all Append() calls
I20260809 09:32:32.251468 23718 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 22534bf98c554d6c8acdd00f8d2a9a29: No bootstrap required, opened a new log
I20260809 09:32:32.255270 23718 raft_consensus.cc:348] T 00000000000000000000000000000000 P 22534bf98c554d6c8acdd00f8d2a9a29 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "22534bf98c554d6c8acdd00f8d2a9a29" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 41195 } }
I20260809 09:32:32.255604 23718 raft_consensus.cc:374] T 00000000000000000000000000000000 P 22534bf98c554d6c8acdd00f8d2a9a29 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
Aug 09 09:32:32 dist-test-slave-7kzw krb5kdc[23584](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786267952, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.125.190@KRBTEST.COM
I20260809 09:32:32.255685 23718 raft_consensus.cc:729] T 00000000000000000000000000000000 P 22534bf98c554d6c8acdd00f8d2a9a29 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 22534bf98c554d6c8acdd00f8d2a9a29, State: Initialized, Role: FOLLOWER
I20260809 09:32:32.256165 23718 consensus_queue.cc:260] T 00000000000000000000000000000000 P 22534bf98c554d6c8acdd00f8d2a9a29 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "22534bf98c554d6c8acdd00f8d2a9a29" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 41195 } }
I20260809 09:32:32.256356 23718 raft_consensus.cc:388] T 00000000000000000000000000000000 P 22534bf98c554d6c8acdd00f8d2a9a29 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 09:32:32.256456 23718 raft_consensus.cc:482] T 00000000000000000000000000000000 P 22534bf98c554d6c8acdd00f8d2a9a29 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 09:32:32.256594 23718 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 22534bf98c554d6c8acdd00f8d2a9a29 [term 0 FOLLOWER]: Advancing to term 1
I20260809 09:32:32.257550 23718 raft_consensus.cc:504] T 00000000000000000000000000000000 P 22534bf98c554d6c8acdd00f8d2a9a29 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "22534bf98c554d6c8acdd00f8d2a9a29" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 41195 } }
I20260809 09:32:32.257928 23718 leader_election.cc:302] T 00000000000000000000000000000000 P 22534bf98c554d6c8acdd00f8d2a9a29 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 22534bf98c554d6c8acdd00f8d2a9a29; no voters:
I20260809 09:32:32.258318 23718 leader_election.cc:288] T 00000000000000000000000000000000 P 22534bf98c554d6c8acdd00f8d2a9a29 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 09:32:32.258406 23727 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 22534bf98c554d6c8acdd00f8d2a9a29 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 09:32:32.258647 23727 raft_consensus.cc:686] T 00000000000000000000000000000000 P 22534bf98c554d6c8acdd00f8d2a9a29 [term 1 LEADER]: Becoming Leader. State: Replica: 22534bf98c554d6c8acdd00f8d2a9a29, State: Running, Role: LEADER
I20260809 09:32:32.258965 23727 consensus_queue.cc:237] T 00000000000000000000000000000000 P 22534bf98c554d6c8acdd00f8d2a9a29 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "22534bf98c554d6c8acdd00f8d2a9a29" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 41195 } }
I20260809 09:32:32.259752 23718 sys_catalog.cc:564] T 00000000000000000000000000000000 P 22534bf98c554d6c8acdd00f8d2a9a29 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 09:32:32.260262 23727 sys_catalog.cc:455] T 00000000000000000000000000000000 P 22534bf98c554d6c8acdd00f8d2a9a29 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 22534bf98c554d6c8acdd00f8d2a9a29. Latest consensus state: current_term: 1 leader_uuid: "22534bf98c554d6c8acdd00f8d2a9a29" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "22534bf98c554d6c8acdd00f8d2a9a29" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 41195 } } }
I20260809 09:32:32.260385 23727 sys_catalog.cc:458] T 00000000000000000000000000000000 P 22534bf98c554d6c8acdd00f8d2a9a29 [sys.catalog]: This master's current role is: LEADER
I20260809 09:32:32.260751 23728 sys_catalog.cc:455] T 00000000000000000000000000000000 P 22534bf98c554d6c8acdd00f8d2a9a29 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "22534bf98c554d6c8acdd00f8d2a9a29" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "22534bf98c554d6c8acdd00f8d2a9a29" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 41195 } } }
I20260809 09:32:32.260887 23728 sys_catalog.cc:458] T 00000000000000000000000000000000 P 22534bf98c554d6c8acdd00f8d2a9a29 [sys.catalog]: This master's current role is: LEADER
I20260809 09:32:32.263119 23725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:32.247527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38990 (local address 127.20.125.190:41195)
0809 09:32:32.248027 (+ 500us) server_negotiation.cc:206] Beginning negotiation
0809 09:32:32.248043 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 09:32:32.248082 (+ 39us) server_negotiation.cc:407] Connection header received
0809 09:32:32.248864 (+ 782us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:32:32.248884 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:32:32.249231 (+ 347us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:32:32.249590 (+ 359us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:32:32.250536 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:32.251668 (+ 1132us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:32.252612 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:32.253133 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:32.253322 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:32.256388 (+ 3066us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:32:32.256394 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:32:32.256413 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:32:32.256448 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:32:32.258787 (+ 2339us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:32:32.259363 (+ 576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:32:32.259368 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:32:32.259373 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:32:32.259491 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:32:32.259842 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:32:32.259848 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:32:32.259849 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:32:32.260420 (+ 571us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:32:32.261417 (+ 997us) server_negotiation.cc:1035] Waiting for connection context
0809 09:32:32.261594 (+ 177us) server_negotiation.cc:299] Negotiation successful
0809 09:32:32.261941 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":347,"thread_start_us":165,"threads_started":1}
W20260809 09:32:32.265465 23750 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 22534bf98c554d6c8acdd00f8d2a9a29: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 09:32:32.265551 23750 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 09:32:32.265637 23735 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 09:32:32.266372 23735 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 09:32:32.272382 23735 catalog_manager.cc:1347] Generated new cluster ID: e391015bd2864a8ab2f56e241dff1aaa
I20260809 09:32:32.272460 23735 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 09:32:32.280771 23735 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 09:32:32.281775 23735 catalog_manager.cc:1504] Loading token signing keys...
I20260809 09:32:32.291667 23735 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 22534bf98c554d6c8acdd00f8d2a9a29: Generated new TSK 0
I20260809 09:32:32.292452 23735 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.20.125.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.125.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.125.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.129.keytab.
Entry for principal kudu/127.20.125.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.129.keytab.
I20260809 09:32:32.328210 20982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
/tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786267941944493-20982-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786267941944493-20982-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786267941944493-20982-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786267941944493-20982-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.125.129:0
--local_ip_for_outbound_sockets=127.20.125.129
--webserver_interface=127.20.125.129
--webserver_port=0
--tserver_master_addrs=127.20.125.190:41195
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.129.keytab
--principal=kudu/127.20.125.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.125.148:33107
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786267941944493-20982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786267941944493-20982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786267941944493-20982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 09:32:32.446363 23757 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 09:32:32.446769 23757 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 09:32:32.446864 23757 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 09:32:32.450953 23757 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 09:32:32.451040 23757 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 09:32:32.451117 23757 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260809 09:32:32.451161 23757 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 09:32:32.451216 23757 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.125.129
I20260809 09:32:32.456557 23757 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.125.148:33107
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786267941944493-20982-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786267941944493-20982-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.129.keytab
--principal=kudu/127.20.125.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.125.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786267941944493-20982-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.125.129
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.20.125.190:41195
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23757
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.125.129
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786267941944493-20982-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 08:38:55 UTC on ccycloud.quasar-njjleu.root.comops.site
build id 4575935
I20260809 09:32:32.457655 23757 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 09:32:32.458771 23757 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 09:32:32.469082 23757 server_base.cc:1034] running on GCE node
W20260809 09:32:32.469029 23764 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:32:32.469018 23765 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:32:32.469023 23767 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:32:32.469751 23757 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 09:32:32.470456 23757 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 09:32:32.471747 23757 hybrid_clock.cc:648] HybridClock initialized: now 1786267952471712 us; error 49 us; skew 500 ppm
Aug 09 09:32:32 dist-test-slave-7kzw krb5kdc[23584](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786267952, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 09:32:32.474802 23757 init.cc:377] Logged in from keytab as kudu/127.20.125.129@KRBTEST.COM (short username kudu)
I20260809 09:32:32.475750 23757 webserver.cc:460] Webserver started at http://127.20.125.129:38257/ using document root <none> and password file <none>
I20260809 09:32:32.476339 23757 fs_manager.cc:359] Metadata directory not provided
I20260809 09:32:32.476434 23757 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 09:32:32.476678 23757 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 09:32:32.478842 23757 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786267941944493-20982-0/minicluster-data/ts-0/data/instance:
uuid: "ff6decf4b31348898adbb75faaab9faf"
format_stamp: "Formatted at 2026-08-09 09:32:32 on dist-test-slave-7kzw"
server_key: "6dc159bddbcf3740d06f483496f110f0"
server_key_iv: "281536bf1ef72fe0dde239589d41c2bf"
server_key_version: "encryptionkey@0"
I20260809 09:32:32.479417 23757 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786267941944493-20982-0/minicluster-data/ts-0/wal/instance:
uuid: "ff6decf4b31348898adbb75faaab9faf"
format_stamp: "Formatted at 2026-08-09 09:32:32 on dist-test-slave-7kzw"
server_key: "6dc159bddbcf3740d06f483496f110f0"
server_key_iv: "281536bf1ef72fe0dde239589d41c2bf"
server_key_version: "encryptionkey@0"
I20260809 09:32:32.483363 23757 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260809 09:32:32.485877 23777 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 09:32:32.487445 23757 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 09:32:32.487648 23757 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786267941944493-20982-0/minicluster-data/ts-0/data,/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786267941944493-20982-0/minicluster-data/ts-0/wal
uuid: "ff6decf4b31348898adbb75faaab9faf"
format_stamp: "Formatted at 2026-08-09 09:32:32 on dist-test-slave-7kzw"
server_key: "6dc159bddbcf3740d06f483496f110f0"
server_key_iv: "281536bf1ef72fe0dde239589d41c2bf"
server_key_version: "encryptionkey@0"
I20260809 09:32:32.487787 23757 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786267941944493-20982-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786267941944493-20982-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786267941944493-20982-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 09:32:32.499686 23757 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 09:32:32.503415 23757 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 09:32:32.503650 23757 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 09:32:32.505283 23757 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 09:32:32.505366 23757 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 09:32:32.505434 23757 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 09:32:32.505475 23757 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 09:32:32 dist-test-slave-7kzw krb5kdc[23584](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786267952, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.129@KRBTEST.COM for kudu/127.20.125.190@KRBTEST.COM
I20260809 09:32:32.518199 23757 rpc_server.cc:307] RPC server started. Bound to: 127.20.125.129:37361
I20260809 09:32:32.518234 23947 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.125.129:37361 every 8 connection(s)
I20260809 09:32:32.519477 23757 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786267941944493-20982-0/minicluster-data/ts-0/data/info.pb
I20260809 09:32:32.520380 23725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:32.506095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.129:33911 (local address 127.20.125.190:41195)
0809 09:32:32.506218 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0809 09:32:32.506222 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 09:32:32.507476 (+ 1254us) server_negotiation.cc:407] Connection header received
0809 09:32:32.508535 (+ 1059us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:32:32.508539 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:32:32.508602 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:32:32.508690 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:32:32.510211 (+ 1521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:32.511010 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:32.512317 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:32.512565 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:32.512677 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:32.516000 (+ 3323us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:32:32.516006 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:32:32.516012 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:32:32.516052 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:32:32.517957 (+ 1905us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:32:32.518611 (+ 654us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:32:32.518615 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:32:32.518617 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:32:32.518722 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:32:32.519101 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:32:32.519105 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:32:32.519107 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:32:32.519358 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:32:32.519483 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0809 09:32:32.520064 (+ 581us) server_negotiation.cc:299] Negotiation successful
0809 09:32:32.520196 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260809 09:32:32.521241 23788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:32.506736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:41195 (local address 127.20.125.129:33911)
0809 09:32:32.507329 (+ 593us) negotiation.cc:107] Waiting for socket to connect
0809 09:32:32.507374 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 09:32:32.508322 (+ 948us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:32:32.508993 (+ 671us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:32:32.509002 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:32:32.509532 (+ 530us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:32:32.510038 (+ 506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:32:32.510052 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:32:32.511151 (+ 1099us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:32:32.511155 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:32:32.512174 (+ 1019us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:32:32.512185 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:32:32.512733 (+ 548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:32:32.512738 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:32:32.512913 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:32.513503 (+ 590us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 09:32:32.513530 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:32:32.515817 (+ 2287us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 09:32:32.518085 (+ 2268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:32:32.518091 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:32:32.518105 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:32:32.518505 (+ 400us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:32:32.518829 (+ 324us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:32:32.518831 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:32:32.518832 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:32:32.518966 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:32:32.519476 (+ 510us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 09:32:32.519481 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 09:32:32.519791 (+ 310us) client_negotiation.cc:769] Sending connection context
0809 09:32:32.519992 (+ 201us) client_negotiation.cc:240] Negotiation successful
0809 09:32:32.520312 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":321,"thread_start_us":154,"threads_started":1}
I20260809 09:32:32.522704 23950 heartbeater.cc:347] Connected to a master server at 127.20.125.190:41195
I20260809 09:32:32.522864 23950 heartbeater.cc:464] Registering TS with master...
I20260809 09:32:32.523479 23950 heartbeater.cc:511] Master 127.20.125.190:41195 requested a full tablet report, sending...
I20260809 09:32:32.525239 23644 ts_manager.cc:194] Registered new tserver with Master: ff6decf4b31348898adbb75faaab9faf (127.20.125.129:37361)
I20260809 09:32:32.525903 20982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu as pid 23757
I20260809 09:32:32.526046 20982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786267941944493-20982-0/minicluster-data/ts-0/wal/instance
I20260809 09:32:32.526355 20982 external_mini_cluster.cc:1428] Setting key 47eb7397f1e51d6afa45621ebcdb3ada
I20260809 09:32:32.526612 23644 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.125.129@KRBTEST.COM'} at 127.20.125.129:33911
I20260809 09:32:32.534875 23959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:32.525108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:41195 (local address 127.20.125.129:43117)
0809 09:32:32.525421 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0809 09:32:32.525438 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 09:32:32.525538 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:32:32.526888 (+ 1350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:32:32.526890 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:32:32.527103 (+ 213us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:32:32.527245 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:32:32.527254 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:32:32.528254 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:32:32.528259 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:32:32.529242 (+ 983us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:32:32.529256 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:32:32.529861 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:32:32.529866 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:32:32.529992 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:32.530555 (+ 563us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 09:32:32.530567 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:32:32.530903 (+ 336us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 09:32:32.533037 (+ 2134us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:32:32.533040 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:32:32.533042 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:32:32.533271 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:32:32.533672 (+ 401us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:32:32.533676 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:32:32.533678 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:32:32.533798 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:32:32.534327 (+ 529us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 09:32:32.534330 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 09:32:32.534417 (+ 87us) client_negotiation.cc:769] Sending connection context
0809 09:32:32.534597 (+ 180us) client_negotiation.cc:240] Negotiation successful
0809 09:32:32.534723 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":89,"threads_started":1}
I20260809 09:32:32.535115 23725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:32.525243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.129:43117 (local address 127.20.125.190:41195)
0809 09:32:32.526582 (+ 1339us) server_negotiation.cc:206] Beginning negotiation
0809 09:32:32.526586 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 09:32:32.526602 (+ 16us) server_negotiation.cc:407] Connection header received
0809 09:32:32.526639 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:32:32.526642 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:32:32.526696 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:32:32.526779 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:32:32.527387 (+ 608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:32.528097 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:32.529398 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:32.529715 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:32.529865 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:32.531051 (+ 1186us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:32:32.531058 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:32:32.531063 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:32:32.531098 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:32:32.532894 (+ 1796us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:32:32.533399 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:32:32.533407 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:32:32.533412 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:32:32.533523 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:32:32.533916 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:32:32.533922 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:32:32.533926 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:32:32.534186 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:32:32.534310 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0809 09:32:32.534699 (+ 389us) server_negotiation.cc:299] Negotiation successful
0809 09:32:32.534855 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1263}
WARNING: no policy specified for kudu/127.20.125.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.125.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.125.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.130.keytab.
Entry for principal kudu/127.20.125.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.130.keytab.
I20260809 09:32:32.557734 20982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
/tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786267941944493-20982-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786267941944493-20982-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786267941944493-20982-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786267941944493-20982-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.125.130:0
--local_ip_for_outbound_sockets=127.20.125.130
--webserver_interface=127.20.125.130
--webserver_port=0
--tserver_master_addrs=127.20.125.190:41195
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.130.keytab
--principal=kudu/127.20.125.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.125.148:33107
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786267941944493-20982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786267941944493-20982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786267941944493-20982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 09:32:32.676939 23965 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 09:32:32.677225 23965 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 09:32:32.677274 23965 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 09:32:32.681505 23965 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 09:32:32.681582 23965 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 09:32:32.681625 23965 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260809 09:32:32.681640 23965 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 09:32:32.681728 23965 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.125.130
I20260809 09:32:32.687361 23965 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.125.148:33107
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786267941944493-20982-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786267941944493-20982-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.130.keytab
--principal=kudu/127.20.125.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.125.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786267941944493-20982-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.125.130
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.20.125.190:41195
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23965
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.125.130
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786267941944493-20982-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 08:38:55 UTC on ccycloud.quasar-njjleu.root.comops.site
build id 4575935
I20260809 09:32:32.688496 23965 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 09:32:32.689582 23965 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 09:32:32.699620 23975 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:32:32.699627 23978 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:32:32.699908 23965 server_base.cc:1034] running on GCE node
W20260809 09:32:32.699620 23976 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:32:32.700482 23965 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 09:32:32.701150 23965 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 09:32:32.702322 23965 hybrid_clock.cc:648] HybridClock initialized: now 1786267952702293 us; error 43 us; skew 500 ppm
Aug 09 09:32:32 dist-test-slave-7kzw krb5kdc[23584](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786267952, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 09:32:32.705212 23965 init.cc:377] Logged in from keytab as kudu/127.20.125.130@KRBTEST.COM (short username kudu)
I20260809 09:32:32.706187 23965 webserver.cc:460] Webserver started at http://127.20.125.130:36091/ using document root <none> and password file <none>
I20260809 09:32:32.706981 23965 fs_manager.cc:359] Metadata directory not provided
I20260809 09:32:32.707052 23965 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 09:32:32.707340 23965 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 09:32:32.709434 23965 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786267941944493-20982-0/minicluster-data/ts-1/data/instance:
uuid: "ed89bfa60b114ee6a11787f6913bf1be"
format_stamp: "Formatted at 2026-08-09 09:32:32 on dist-test-slave-7kzw"
server_key: "2fdf9534bfdc604d668f3c0a599bb44c"
server_key_iv: "2f9be00d67fb392af2f80729b9f1dd26"
server_key_version: "encryptionkey@0"
I20260809 09:32:32.710045 23965 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786267941944493-20982-0/minicluster-data/ts-1/wal/instance:
uuid: "ed89bfa60b114ee6a11787f6913bf1be"
format_stamp: "Formatted at 2026-08-09 09:32:32 on dist-test-slave-7kzw"
server_key: "2fdf9534bfdc604d668f3c0a599bb44c"
server_key_iv: "2f9be00d67fb392af2f80729b9f1dd26"
server_key_version: "encryptionkey@0"
I20260809 09:32:32.713868 23965 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260809 09:32:32.716466 23992 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 09:32:32.717882 23965 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 09:32:32.718016 23965 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786267941944493-20982-0/minicluster-data/ts-1/data,/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786267941944493-20982-0/minicluster-data/ts-1/wal
uuid: "ed89bfa60b114ee6a11787f6913bf1be"
format_stamp: "Formatted at 2026-08-09 09:32:32 on dist-test-slave-7kzw"
server_key: "2fdf9534bfdc604d668f3c0a599bb44c"
server_key_iv: "2f9be00d67fb392af2f80729b9f1dd26"
server_key_version: "encryptionkey@0"
I20260809 09:32:32.718135 23965 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786267941944493-20982-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786267941944493-20982-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786267941944493-20982-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 09:32:32.731942 23965 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 09:32:32.735447 23965 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 09:32:32.735667 23965 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 09:32:32.737249 23965 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 09:32:32.737313 23965 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 09:32:32.737358 23965 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 09:32:32.737375 23965 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 09:32:32 dist-test-slave-7kzw krb5kdc[23584](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786267952, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.130@KRBTEST.COM for kudu/127.20.125.190@KRBTEST.COM
I20260809 09:32:32.748718 23965 rpc_server.cc:307] RPC server started. Bound to: 127.20.125.130:37329
I20260809 09:32:32.748798 24167 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.125.130:37329 every 8 connection(s)
I20260809 09:32:32.749635 23965 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786267941944493-20982-0/minicluster-data/ts-1/data/info.pb
I20260809 09:32:32.752518 23725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:32.738152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.130:58933 (local address 127.20.125.190:41195)
0809 09:32:32.738304 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0809 09:32:32.738309 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 09:32:32.739514 (+ 1205us) server_negotiation.cc:407] Connection header received
0809 09:32:32.740665 (+ 1151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:32:32.740668 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:32:32.740745 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:32:32.740860 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:32:32.742349 (+ 1489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:32.742997 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:32.744769 (+ 1772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:32.745201 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:32.745344 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:32.748435 (+ 3091us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:32:32.748442 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:32:32.748446 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:32:32.748484 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:32:32.750287 (+ 1803us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:32:32.750906 (+ 619us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:32:32.750909 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:32:32.750911 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:32:32.751009 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:32:32.751371 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:32:32.751374 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:32:32.751376 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:32:32.751584 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:32:32.751696 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0809 09:32:32.752206 (+ 510us) server_negotiation.cc:299] Negotiation successful
0809 09:32:32.752339 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260809 09:32:32.753319 24004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:32.738815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:41195 (local address 127.20.125.130:58933)
0809 09:32:32.739338 (+ 523us) negotiation.cc:107] Waiting for socket to connect
0809 09:32:32.739394 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0809 09:32:32.740409 (+ 1015us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:32:32.741088 (+ 679us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:32:32.741095 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:32:32.741561 (+ 466us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:32:32.742155 (+ 594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:32:32.742193 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:32:32.743134 (+ 941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:32:32.743137 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:32:32.744606 (+ 1469us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:32:32.744617 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:32:32.745396 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:32:32.745403 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:32:32.745622 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:32.746309 (+ 687us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 09:32:32.746326 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:32:32.748193 (+ 1867us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 09:32:32.750393 (+ 2200us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:32:32.750400 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:32:32.750414 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:32:32.750789 (+ 375us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:32:32.751094 (+ 305us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:32:32.751099 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:32:32.751101 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:32:32.751258 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:32:32.751698 (+ 440us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 09:32:32.751704 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 09:32:32.751956 (+ 252us) client_negotiation.cc:769] Sending connection context
0809 09:32:32.752143 (+ 187us) client_negotiation.cc:240] Negotiation successful
0809 09:32:32.752473 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":311,"thread_start_us":133,"threads_started":1}
I20260809 09:32:32.754645 24168 heartbeater.cc:347] Connected to a master server at 127.20.125.190:41195
I20260809 09:32:32.754818 24168 heartbeater.cc:464] Registering TS with master...
I20260809 09:32:32.755199 20982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu as pid 23965
I20260809 09:32:32.755326 20982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786267941944493-20982-0/minicluster-data/ts-1/wal/instance
I20260809 09:32:32.755442 24168 heartbeater.cc:511] Master 127.20.125.190:41195 requested a full tablet report, sending...
I20260809 09:32:32.755668 20982 external_mini_cluster.cc:1428] Setting key 05f5bf1e95f64a674ca5162073b19e66
I20260809 09:32:32.756647 23646 ts_manager.cc:194] Registered new tserver with Master: ed89bfa60b114ee6a11787f6913bf1be (127.20.125.130:37329)
I20260809 09:32:32.757382 23646 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.125.130@KRBTEST.COM'} at 127.20.125.130:58933
I20260809 09:32:32.765976 23725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:32.757172 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.130:54365 (local address 127.20.125.190:41195)
0809 09:32:32.757400 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0809 09:32:32.757404 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 09:32:32.757895 (+ 491us) server_negotiation.cc:407] Connection header received
0809 09:32:32.758594 (+ 699us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:32:32.758597 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:32:32.758645 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:32:32.758804 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:32:32.759265 (+ 461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:32.759947 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:32.760780 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:32.761114 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:32.761234 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:32.762646 (+ 1412us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:32:32.762649 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:32:32.762651 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:32:32.762678 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:32:32.764105 (+ 1427us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:32:32.764642 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:32:32.764644 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:32:32.764646 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:32:32.764748 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:32:32.765110 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:32:32.765112 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:32:32.765114 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:32:32.765336 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:32:32.765429 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0809 09:32:32.765751 (+ 322us) server_negotiation.cc:299] Negotiation successful
0809 09:32:32.765850 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152}
I20260809 09:32:32.766115 24176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:32.757014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:41195 (local address 127.20.125.130:54365)
0809 09:32:32.757794 (+ 780us) negotiation.cc:107] Waiting for socket to connect
0809 09:32:32.757809 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 09:32:32.757916 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:32:32.758834 (+ 918us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:32:32.758837 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:32:32.759052 (+ 215us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:32:32.759146 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:32:32.759151 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:32:32.760090 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:32:32.760092 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:32:32.760637 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:32:32.760645 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:32:32.761250 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:32:32.761254 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:32:32.761363 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:32.761946 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 09:32:32.761966 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:32:32.762527 (+ 561us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 09:32:32.764222 (+ 1695us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:32:32.764229 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:32:32.764234 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:32:32.764529 (+ 295us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:32:32.764846 (+ 317us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:32:32.764852 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:32:32.764857 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:32:32.764971 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:32:32.765454 (+ 483us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 09:32:32.765460 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 09:32:32.765573 (+ 113us) client_negotiation.cc:769] Sending connection context
0809 09:32:32.765692 (+ 119us) client_negotiation.cc:240] Negotiation successful
0809 09:32:32.765869 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":695,"thread_start_us":120,"threads_started":1}
WARNING: no policy specified for kudu/127.20.125.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.125.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.125.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.131.keytab.
Entry for principal kudu/127.20.125.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.131.keytab.
I20260809 09:32:32.788089 20982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
/tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786267941944493-20982-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786267941944493-20982-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786267941944493-20982-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786267941944493-20982-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.125.131:0
--local_ip_for_outbound_sockets=127.20.125.131
--webserver_interface=127.20.125.131
--webserver_port=0
--tserver_master_addrs=127.20.125.190:41195
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.131.keytab
--principal=kudu/127.20.125.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.125.148:33107
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786267941944493-20982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786267941944493-20982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786267941944493-20982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 09:32:32.903625 24180 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 09:32:32.903941 24180 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 09:32:32.904026 24180 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 09:32:32.908087 24180 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 09:32:32.908190 24180 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 09:32:32.908265 24180 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260809 09:32:32.908309 24180 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 09:32:32.908375 24180 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.125.131
I20260809 09:32:32.914048 24180 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.125.148:33107
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786267941944493-20982-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786267941944493-20982-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.131.keytab
--principal=kudu/127.20.125.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.125.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786267941944493-20982-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.125.131
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.20.125.190:41195
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24180
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.125.131
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786267941944493-20982-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 08:38:55 UTC on ccycloud.quasar-njjleu.root.comops.site
build id 4575935
I20260809 09:32:32.915299 24180 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 09:32:32.916373 24180 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 09:32:32.926301 24187 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:32:32.926384 24192 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:32:32.926725 24180 server_base.cc:1034] running on GCE node
W20260809 09:32:32.926390 24188 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:32:32.927183 24180 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 09:32:32.927883 24180 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 09:32:32.929123 24180 hybrid_clock.cc:648] HybridClock initialized: now 1786267952929081 us; error 50 us; skew 500 ppm
Aug 09 09:32:32 dist-test-slave-7kzw krb5kdc[23584](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786267952, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 09:32:32.932091 24180 init.cc:377] Logged in from keytab as kudu/127.20.125.131@KRBTEST.COM (short username kudu)
I20260809 09:32:32.933022 24180 webserver.cc:460] Webserver started at http://127.20.125.131:46087/ using document root <none> and password file <none>
I20260809 09:32:32.933604 24180 fs_manager.cc:359] Metadata directory not provided
I20260809 09:32:32.933723 24180 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 09:32:32.933965 24180 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 09:32:32.936057 24180 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786267941944493-20982-0/minicluster-data/ts-2/data/instance:
uuid: "768749257e7f437693e5f9a234a3db0c"
format_stamp: "Formatted at 2026-08-09 09:32:32 on dist-test-slave-7kzw"
server_key: "32987a69e47c0b33a3ecf475d98bffeb"
server_key_iv: "8f9ae8d75b09757a8aa6fca270a28551"
server_key_version: "encryptionkey@0"
I20260809 09:32:32.936635 24180 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786267941944493-20982-0/minicluster-data/ts-2/wal/instance:
uuid: "768749257e7f437693e5f9a234a3db0c"
format_stamp: "Formatted at 2026-08-09 09:32:32 on dist-test-slave-7kzw"
server_key: "32987a69e47c0b33a3ecf475d98bffeb"
server_key_iv: "8f9ae8d75b09757a8aa6fca270a28551"
server_key_version: "encryptionkey@0"
I20260809 09:32:32.940411 24180 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260809 09:32:32.942955 24206 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 09:32:32.944204 24180 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 09:32:32.944343 24180 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786267941944493-20982-0/minicluster-data/ts-2/data,/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786267941944493-20982-0/minicluster-data/ts-2/wal
uuid: "768749257e7f437693e5f9a234a3db0c"
format_stamp: "Formatted at 2026-08-09 09:32:32 on dist-test-slave-7kzw"
server_key: "32987a69e47c0b33a3ecf475d98bffeb"
server_key_iv: "8f9ae8d75b09757a8aa6fca270a28551"
server_key_version: "encryptionkey@0"
I20260809 09:32:32.944461 24180 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786267941944493-20982-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786267941944493-20982-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786267941944493-20982-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 09:32:32.962015 24180 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 09:32:32.965428 24180 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 09:32:32.965602 24180 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 09:32:32.967218 24180 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 09:32:32.967285 24180 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 09:32:32.967360 24180 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 09:32:32.967453 24180 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
Aug 09 09:32:32 dist-test-slave-7kzw krb5kdc[23584](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786267952, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.131@KRBTEST.COM for kudu/127.20.125.190@KRBTEST.COM
I20260809 09:32:32.981897 24180 rpc_server.cc:307] RPC server started. Bound to: 127.20.125.131:38953
I20260809 09:32:32.981956 24379 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.125.131:38953 every 8 connection(s)
I20260809 09:32:32.983068 24180 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786267941944493-20982-0/minicluster-data/ts-2/data/info.pb
I20260809 09:32:32.983330 23725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:32.967964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.131:36675 (local address 127.20.125.190:41195)
0809 09:32:32.968106 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0809 09:32:32.968115 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 09:32:32.969926 (+ 1811us) server_negotiation.cc:407] Connection header received
0809 09:32:32.971231 (+ 1305us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:32:32.971235 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:32:32.971306 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:32:32.971424 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:32:32.972842 (+ 1418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:32.973651 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:32.974954 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:32.975205 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:32.975337 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:32.978545 (+ 3208us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:32:32.978550 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:32:32.978553 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:32:32.978645 (+ 92us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:32:32.980820 (+ 2175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:32:32.981512 (+ 692us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:32:32.981517 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:32:32.981519 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:32:32.981630 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:32:32.982095 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:32:32.982098 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:32:32.982100 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:32:32.982321 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:32:32.982406 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0809 09:32:32.983093 (+ 687us) server_negotiation.cc:299] Negotiation successful
0809 09:32:32.983203 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260809 09:32:32.984453 24219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:32.969113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:41195 (local address 127.20.125.131:36675)
0809 09:32:32.969728 (+ 615us) negotiation.cc:107] Waiting for socket to connect
0809 09:32:32.969788 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0809 09:32:32.970986 (+ 1198us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:32:32.971612 (+ 626us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:32:32.971621 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:32:32.972159 (+ 538us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:32:32.972655 (+ 496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:32:32.972670 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:32:32.973791 (+ 1121us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:32:32.973795 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:32:32.974813 (+ 1018us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:32:32.974825 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:32:32.975393 (+ 568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:32:32.975398 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:32:32.975788 (+ 390us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:32.976537 (+ 749us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 09:32:32.976554 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:32:32.978309 (+ 1755us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 09:32:32.980969 (+ 2660us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:32:32.980975 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:32:32.980990 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:32:32.981391 (+ 401us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:32:32.981734 (+ 343us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:32:32.981737 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:32:32.981739 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:32:32.981954 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:32:32.982441 (+ 487us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 09:32:32.982447 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 09:32:32.982814 (+ 367us) client_negotiation.cc:769] Sending connection context
0809 09:32:32.983032 (+ 218us) client_negotiation.cc:240] Negotiation successful
0809 09:32:32.983378 (+ 346us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":370,"thread_start_us":152,"threads_started":1}
I20260809 09:32:32.985766 24380 heartbeater.cc:347] Connected to a master server at 127.20.125.190:41195
I20260809 09:32:32.985947 24380 heartbeater.cc:464] Registering TS with master...
I20260809 09:32:32.986507 20982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu as pid 24180
I20260809 09:32:32.986568 24380 heartbeater.cc:511] Master 127.20.125.190:41195 requested a full tablet report, sending...
I20260809 09:32:32.986658 20982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786267941944493-20982-0/minicluster-data/ts-2/wal/instance
I20260809 09:32:32.987002 20982 external_mini_cluster.cc:1428] Setting key 18b25043ce56211989c6de5ff3a1d5c1
I20260809 09:32:32.987692 23644 ts_manager.cc:194] Registered new tserver with Master: 768749257e7f437693e5f9a234a3db0c (127.20.125.131:38953)
I20260809 09:32:32.988284 23644 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.125.131@KRBTEST.COM'} at 127.20.125.131:36675
I20260809 09:32:32.990736 20982 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 09:32:33.001700 23725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:32.988626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.131:38815 (local address 127.20.125.190:41195)
0809 09:32:32.988781 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0809 09:32:32.988786 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 09:32:32.988801 (+ 15us) server_negotiation.cc:407] Connection header received
0809 09:32:32.988836 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:32:32.988839 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:32:32.988884 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:32:32.988974 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:32:32.989813 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:32.990585 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:32.995665 (+ 5080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:32.996107 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:32.996251 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:32.997468 (+ 1217us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:32:32.997475 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:32:32.997482 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:32:32.997519 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:32:32.999609 (+ 2090us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:32:33.000175 (+ 566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:32:33.000178 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:32:33.000180 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:32:33.000263 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:32:33.000646 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:32:33.000649 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:32:33.000651 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:32:33.000848 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:32:33.000933 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0809 09:32:33.001424 (+ 491us) server_negotiation.cc:299] Negotiation successful
0809 09:32:33.001532 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260809 09:32:33.001927 24390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:32.988075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:41195 (local address 127.20.125.131:38815)
0809 09:32:32.988350 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0809 09:32:32.988364 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 09:32:32.988454 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:32:32.989323 (+ 869us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:32:32.989326 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:32:32.989553 (+ 227us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:32:32.989668 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:32:32.989675 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:32:32.994690 (+ 5015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:32:32.994694 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:32:32.995517 (+ 823us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:32:32.995528 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:32:32.996259 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:32:32.996263 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:32:32.996339 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:32.996946 (+ 607us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 09:32:32.996958 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:32:32.997303 (+ 345us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 09:32:32.999763 (+ 2460us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:32:32.999767 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:32:32.999769 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:32:33.000055 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:32:33.000405 (+ 350us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:32:33.000411 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:32:33.000416 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:32:33.000531 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:32:33.000974 (+ 443us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 09:32:33.000977 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 09:32:33.001167 (+ 190us) client_negotiation.cc:769] Sending connection context
0809 09:32:33.001357 (+ 190us) client_negotiation.cc:240] Negotiation successful
0809 09:32:33.001597 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":108,"threads_started":1}
Aug 09 09:32:33 dist-test-slave-7kzw krb5kdc[23584](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786267953, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 09:32:33 dist-test-slave-7kzw krb5kdc[23584](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786267953, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.125.190@KRBTEST.COM
I20260809 09:32:33.026912 23725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:33.017511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39006 (local address 127.20.125.190:41195)
0809 09:32:33.017760 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0809 09:32:33.017764 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 09:32:33.017780 (+ 16us) server_negotiation.cc:407] Connection header received
0809 09:32:33.017827 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:32:33.017829 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:32:33.017874 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:32:33.017955 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:32:33.018782 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:33.019324 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:33.020208 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:33.020428 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:33.020523 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:33.023418 (+ 2895us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:32:33.023421 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:32:33.023424 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:32:33.023452 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:32:33.025085 (+ 1633us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:32:33.025567 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:32:33.025571 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:32:33.025572 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:32:33.025668 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:32:33.025953 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:32:33.025956 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:32:33.025957 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:32:33.026171 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:32:33.026279 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0809 09:32:33.026581 (+ 302us) server_negotiation.cc:299] Negotiation successful
0809 09:32:33.026694 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":157}
I20260809 09:32:33.030352 23644 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39006:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 09:32:33.033406 23644 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 09:32:33.049707 24408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:33.042896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.129:37361 (local address 127.0.0.1:35230)
0809 09:32:33.044113 (+ 1217us) negotiation.cc:107] Waiting for socket to connect
0809 09:32:33.044127 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 09:32:33.044299 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:32:33.045122 (+ 823us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:32:33.045125 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:32:33.045146 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 09:32:33.045489 (+ 343us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:32:33.045500 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:32:33.047017 (+ 1517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:32:33.047020 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:32:33.047957 (+ 937us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:32:33.047967 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:32:33.049094 (+ 1127us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:32:33.049099 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:32:33.049206 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:33.049256 (+ 50us) client_negotiation.cc:769] Sending connection context
0809 09:32:33.049376 (+ 120us) client_negotiation.cc:240] Negotiation successful
0809 09:32:33.049510 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":662,"mutex_wait_us":190,"thread_start_us":75,"threads_started":1}
I20260809 09:32:33.050266 24413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:33.043841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45356 (local address 127.20.125.131:38953)
0809 09:32:33.044439 (+ 598us) server_negotiation.cc:206] Beginning negotiation
0809 09:32:33.044453 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 09:32:33.044482 (+ 29us) server_negotiation.cc:407] Connection header received
0809 09:32:33.044544 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:32:33.044547 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:32:33.044692 (+ 145us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:32:33.045095 (+ 403us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 09:32:33.045587 (+ 492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:33.046860 (+ 1273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:33.048476 (+ 1616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:33.049225 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:33.049359 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:33.049476 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0809 09:32:33.049987 (+ 511us) server_negotiation.cc:299] Negotiation successful
0809 09:32:33.050092 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":475,"thread_start_us":116,"threads_started":1}
I20260809 09:32:33.050316 24412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:33.043814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48072 (local address 127.20.125.130:37329)
0809 09:32:33.044564 (+ 750us) server_negotiation.cc:206] Beginning negotiation
0809 09:32:33.044575 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 09:32:33.044596 (+ 21us) server_negotiation.cc:407] Connection header received
0809 09:32:33.044677 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:32:33.044682 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:32:33.044905 (+ 223us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:32:33.045037 (+ 132us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 09:32:33.045645 (+ 608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:33.046768 (+ 1123us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:33.048509 (+ 1741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:33.049095 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:33.049661 (+ 566us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:33.049791 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0809 09:32:33.050088 (+ 297us) server_negotiation.cc:299] Negotiation successful
0809 09:32:33.050179 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":655,"thread_start_us":107,"threads_started":1}
I20260809 09:32:33.050314 24403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:33.043023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35230 (local address 127.20.125.129:37361)
0809 09:32:33.043476 (+ 453us) server_negotiation.cc:206] Beginning negotiation
0809 09:32:33.043483 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 09:32:33.044236 (+ 753us) server_negotiation.cc:407] Connection header received
0809 09:32:33.044447 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:32:33.044453 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:32:33.044673 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:32:33.044924 (+ 251us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 09:32:33.045714 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:33.046859 (+ 1145us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:33.048166 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:33.048950 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:33.049089 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:33.049229 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0809 09:32:33.050087 (+ 858us) server_negotiation.cc:299] Negotiation successful
0809 09:32:33.050187 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":349,"thread_start_us":123,"threads_started":1}
I20260809 09:32:33.049793 24402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:33.043339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.131:38953 (local address 127.0.0.1:45356)
0809 09:32:33.044058 (+ 719us) negotiation.cc:107] Waiting for socket to connect
0809 09:32:33.044113 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0809 09:32:33.044262 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:32:33.045094 (+ 832us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:32:33.045100 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:32:33.045125 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 09:32:33.045428 (+ 303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:32:33.045440 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:32:33.046985 (+ 1545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:32:33.046989 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:32:33.048339 (+ 1350us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:32:33.048351 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:32:33.049344 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:32:33.049349 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:32:33.049434 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:33.049458 (+ 24us) client_negotiation.cc:769] Sending connection context
0809 09:32:33.049539 (+ 81us) client_negotiation.cc:240] Negotiation successful
0809 09:32:33.049612 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":543,"mutex_wait_us":174,"spinlock_wait_cycles":3968,"thread_start_us":100,"threads_started":1}
I20260809 09:32:33.050264 24410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:33.043634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.130:37329 (local address 127.0.0.1:48072)
0809 09:32:33.044110 (+ 476us) negotiation.cc:107] Waiting for socket to connect
0809 09:32:33.044126 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 09:32:33.044360 (+ 234us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:32:33.045171 (+ 811us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:32:33.045173 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:32:33.045187 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 09:32:33.045516 (+ 329us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:32:33.045524 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:32:33.046881 (+ 1357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:32:33.046886 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:32:33.048381 (+ 1495us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:32:33.048392 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:32:33.049863 (+ 1471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:32:33.049868 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:32:33.049961 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:33.049982 (+ 21us) client_negotiation.cc:769] Sending connection context
0809 09:32:33.050058 (+ 76us) client_negotiation.cc:240] Negotiation successful
0809 09:32:33.050125 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":400,"mutex_wait_us":187,"thread_start_us":232,"threads_started":1}
I20260809 09:32:33.053483 23851 tablet_service.cc:1467] Processing CreateTablet for tablet daa39432dbc14ab198ef697006127abd (DEFAULT_TABLE table=test-table [id=856cdbf2690a418f94d612f9abac6c84]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 09:32:33.053483 24288 tablet_service.cc:1467] Processing CreateTablet for tablet daa39432dbc14ab198ef697006127abd (DEFAULT_TABLE table=test-table [id=856cdbf2690a418f94d612f9abac6c84]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 09:32:33.053483 24067 tablet_service.cc:1467] Processing CreateTablet for tablet daa39432dbc14ab198ef697006127abd (DEFAULT_TABLE table=test-table [id=856cdbf2690a418f94d612f9abac6c84]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 09:32:33.054692 24288 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet daa39432dbc14ab198ef697006127abd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 09:32:33.054692 24067 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet daa39432dbc14ab198ef697006127abd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 09:32:33.054700 23851 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet daa39432dbc14ab198ef697006127abd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 09:32:33.061636 24414 tablet_bootstrap.cc:492] T daa39432dbc14ab198ef697006127abd P ff6decf4b31348898adbb75faaab9faf: Bootstrap starting.
I20260809 09:32:33.063236 24417 tablet_bootstrap.cc:492] T daa39432dbc14ab198ef697006127abd P 768749257e7f437693e5f9a234a3db0c: Bootstrap starting.
I20260809 09:32:33.064254 24414 tablet_bootstrap.cc:654] T daa39432dbc14ab198ef697006127abd P ff6decf4b31348898adbb75faaab9faf: Neither blocks nor log segments found. Creating new log.
I20260809 09:32:33.065266 24414 log.cc:824] T daa39432dbc14ab198ef697006127abd P ff6decf4b31348898adbb75faaab9faf: Log is configured to *not* fsync() on all Append() calls
I20260809 09:32:33.065464 24417 tablet_bootstrap.cc:654] T daa39432dbc14ab198ef697006127abd P 768749257e7f437693e5f9a234a3db0c: Neither blocks nor log segments found. Creating new log.
I20260809 09:32:33.066246 24417 log.cc:824] T daa39432dbc14ab198ef697006127abd P 768749257e7f437693e5f9a234a3db0c: Log is configured to *not* fsync() on all Append() calls
I20260809 09:32:33.066887 24418 tablet_bootstrap.cc:492] T daa39432dbc14ab198ef697006127abd P ed89bfa60b114ee6a11787f6913bf1be: Bootstrap starting.
I20260809 09:32:33.067489 24414 tablet_bootstrap.cc:492] T daa39432dbc14ab198ef697006127abd P ff6decf4b31348898adbb75faaab9faf: No bootstrap required, opened a new log
I20260809 09:32:33.067698 24414 ts_tablet_manager.cc:1397] T daa39432dbc14ab198ef697006127abd P ff6decf4b31348898adbb75faaab9faf: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260809 09:32:33.068290 24417 tablet_bootstrap.cc:492] T daa39432dbc14ab198ef697006127abd P 768749257e7f437693e5f9a234a3db0c: No bootstrap required, opened a new log
I20260809 09:32:33.068475 24417 ts_tablet_manager.cc:1397] T daa39432dbc14ab198ef697006127abd P 768749257e7f437693e5f9a234a3db0c: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260809 09:32:33.068918 24418 tablet_bootstrap.cc:654] T daa39432dbc14ab198ef697006127abd P ed89bfa60b114ee6a11787f6913bf1be: Neither blocks nor log segments found. Creating new log.
I20260809 09:32:33.069585 24418 log.cc:824] T daa39432dbc14ab198ef697006127abd P ed89bfa60b114ee6a11787f6913bf1be: Log is configured to *not* fsync() on all Append() calls
I20260809 09:32:33.071731 24418 tablet_bootstrap.cc:492] T daa39432dbc14ab198ef697006127abd P ed89bfa60b114ee6a11787f6913bf1be: No bootstrap required, opened a new log
I20260809 09:32:33.071931 24418 ts_tablet_manager.cc:1397] T daa39432dbc14ab198ef697006127abd P ed89bfa60b114ee6a11787f6913bf1be: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260809 09:32:33.072450 24414 raft_consensus.cc:348] T daa39432dbc14ab198ef697006127abd P ff6decf4b31348898adbb75faaab9faf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff6decf4b31348898adbb75faaab9faf" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 37361 } } peers { permanent_uuid: "ed89bfa60b114ee6a11787f6913bf1be" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 37329 } } peers { permanent_uuid: "768749257e7f437693e5f9a234a3db0c" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 38953 } }
I20260809 09:32:33.072450 24417 raft_consensus.cc:348] T daa39432dbc14ab198ef697006127abd P 768749257e7f437693e5f9a234a3db0c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff6decf4b31348898adbb75faaab9faf" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 37361 } } peers { permanent_uuid: "ed89bfa60b114ee6a11787f6913bf1be" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 37329 } } peers { permanent_uuid: "768749257e7f437693e5f9a234a3db0c" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 38953 } }
I20260809 09:32:33.072731 24417 raft_consensus.cc:374] T daa39432dbc14ab198ef697006127abd P 768749257e7f437693e5f9a234a3db0c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 09:32:33.072731 24414 raft_consensus.cc:374] T daa39432dbc14ab198ef697006127abd P ff6decf4b31348898adbb75faaab9faf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 09:32:33.072793 24414 raft_consensus.cc:729] T daa39432dbc14ab198ef697006127abd P ff6decf4b31348898adbb75faaab9faf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff6decf4b31348898adbb75faaab9faf, State: Initialized, Role: FOLLOWER
I20260809 09:32:33.072826 24417 raft_consensus.cc:729] T daa39432dbc14ab198ef697006127abd P 768749257e7f437693e5f9a234a3db0c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 768749257e7f437693e5f9a234a3db0c, State: Initialized, Role: FOLLOWER
I20260809 09:32:33.073302 24414 consensus_queue.cc:260] T daa39432dbc14ab198ef697006127abd P ff6decf4b31348898adbb75faaab9faf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff6decf4b31348898adbb75faaab9faf" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 37361 } } peers { permanent_uuid: "ed89bfa60b114ee6a11787f6913bf1be" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 37329 } } peers { permanent_uuid: "768749257e7f437693e5f9a234a3db0c" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 38953 } }
I20260809 09:32:33.073302 24417 consensus_queue.cc:260] T daa39432dbc14ab198ef697006127abd P 768749257e7f437693e5f9a234a3db0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff6decf4b31348898adbb75faaab9faf" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 37361 } } peers { permanent_uuid: "ed89bfa60b114ee6a11787f6913bf1be" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 37329 } } peers { permanent_uuid: "768749257e7f437693e5f9a234a3db0c" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 38953 } }
I20260809 09:32:33.073948 23950 heartbeater.cc:503] Master 127.20.125.190:41195 was elected leader, sending a full tablet report...
I20260809 09:32:33.074287 24417 ts_tablet_manager.cc:1428] T daa39432dbc14ab198ef697006127abd P 768749257e7f437693e5f9a234a3db0c: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260809 09:32:33.074297 24414 ts_tablet_manager.cc:1428] T daa39432dbc14ab198ef697006127abd P ff6decf4b31348898adbb75faaab9faf: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260809 09:32:33.074615 24380 heartbeater.cc:503] Master 127.20.125.190:41195 was elected leader, sending a full tablet report...
I20260809 09:32:33.075460 24418 raft_consensus.cc:348] T daa39432dbc14ab198ef697006127abd P ed89bfa60b114ee6a11787f6913bf1be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff6decf4b31348898adbb75faaab9faf" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 37361 } } peers { permanent_uuid: "ed89bfa60b114ee6a11787f6913bf1be" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 37329 } } peers { permanent_uuid: "768749257e7f437693e5f9a234a3db0c" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 38953 } }
I20260809 09:32:33.075726 24418 raft_consensus.cc:374] T daa39432dbc14ab198ef697006127abd P ed89bfa60b114ee6a11787f6913bf1be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 09:32:33.075779 24418 raft_consensus.cc:729] T daa39432dbc14ab198ef697006127abd P ed89bfa60b114ee6a11787f6913bf1be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed89bfa60b114ee6a11787f6913bf1be, State: Initialized, Role: FOLLOWER
I20260809 09:32:33.076190 24418 consensus_queue.cc:260] T daa39432dbc14ab198ef697006127abd P ed89bfa60b114ee6a11787f6913bf1be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff6decf4b31348898adbb75faaab9faf" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 37361 } } peers { permanent_uuid: "ed89bfa60b114ee6a11787f6913bf1be" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 37329 } } peers { permanent_uuid: "768749257e7f437693e5f9a234a3db0c" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 38953 } }
I20260809 09:32:33.076968 24168 heartbeater.cc:503] Master 127.20.125.190:41195 was elected leader, sending a full tablet report...
I20260809 09:32:33.077280 24418 ts_tablet_manager.cc:1428] T daa39432dbc14ab198ef697006127abd P ed89bfa60b114ee6a11787f6913bf1be: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
W20260809 09:32:33.234429 24382 tablet.cc:2369] T daa39432dbc14ab198ef697006127abd P 768749257e7f437693e5f9a234a3db0c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 09:32:33.250861 24169 tablet.cc:2369] T daa39432dbc14ab198ef697006127abd P ed89bfa60b114ee6a11787f6913bf1be: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 09:32:33.271394 23952 tablet.cc:2369] T daa39432dbc14ab198ef697006127abd P ff6decf4b31348898adbb75faaab9faf: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 09:32:33.308145 24423 raft_consensus.cc:482] T daa39432dbc14ab198ef697006127abd P 768749257e7f437693e5f9a234a3db0c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 09:32:33.308485 24423 raft_consensus.cc:504] T daa39432dbc14ab198ef697006127abd P 768749257e7f437693e5f9a234a3db0c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff6decf4b31348898adbb75faaab9faf" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 37361 } } peers { permanent_uuid: "ed89bfa60b114ee6a11787f6913bf1be" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 37329 } } peers { permanent_uuid: "768749257e7f437693e5f9a234a3db0c" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 38953 } }
I20260809 09:32:33.309892 24423 leader_election.cc:288] T daa39432dbc14ab198ef697006127abd P 768749257e7f437693e5f9a234a3db0c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ff6decf4b31348898adbb75faaab9faf (127.20.125.129:37361), ed89bfa60b114ee6a11787f6913bf1be (127.20.125.130:37329)
I20260809 09:32:33.314916 24403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:33.310324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.131:42039 (local address 127.20.125.129:37361)
0809 09:32:33.310536 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0809 09:32:33.310542 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 09:32:33.310564 (+ 22us) server_negotiation.cc:407] Connection header received
0809 09:32:33.310769 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:32:33.310776 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:32:33.310837 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:32:33.310930 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 09:32:33.311343 (+ 413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:33.312194 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:33.313597 (+ 1403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:33.314242 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:33.314352 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:33.314438 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0809 09:32:33.314724 (+ 286us) server_negotiation.cc:299] Negotiation successful
0809 09:32:33.314792 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260809 09:32:33.314940 24219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:33.310209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.129:37361 (local address 127.20.125.131:42039)
0809 09:32:33.310374 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0809 09:32:33.310394 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 09:32:33.310621 (+ 227us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:32:33.310972 (+ 351us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:32:33.310975 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:32:33.311017 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 09:32:33.311189 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:32:33.311198 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:32:33.312321 (+ 1123us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:32:33.312332 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:32:33.313460 (+ 1128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:32:33.313470 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:32:33.314377 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:32:33.314381 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:32:33.314528 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:33.314558 (+ 30us) client_negotiation.cc:769] Sending connection context
0809 09:32:33.314699 (+ 141us) client_negotiation.cc:240] Negotiation successful
0809 09:32:33.314781 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":58,"spinlock_wait_cycles":896}
I20260809 09:32:33.315719 24431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:33.310539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.130:37329 (local address 127.20.125.131:40569)
0809 09:32:33.310922 (+ 383us) negotiation.cc:107] Waiting for socket to connect
0809 09:32:33.310937 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 09:32:33.311047 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:32:33.311343 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:32:33.311346 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:32:33.311365 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 09:32:33.311982 (+ 617us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:32:33.311994 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:32:33.313108 (+ 1114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:32:33.313111 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:32:33.314127 (+ 1016us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:32:33.314138 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:32:33.315262 (+ 1124us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:32:33.315267 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:32:33.315362 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:33.315380 (+ 18us) client_negotiation.cc:769] Sending connection context
0809 09:32:33.315483 (+ 103us) client_negotiation.cc:240] Negotiation successful
0809 09:32:33.315556 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":293,"spinlock_wait_cycles":896,"thread_start_us":150,"threads_started":1}
I20260809 09:32:33.315708 24412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:33.310717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.131:40569 (local address 127.20.125.130:37329)
0809 09:32:33.310886 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0809 09:32:33.310912 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0809 09:32:33.311018 (+ 106us) server_negotiation.cc:407] Connection header received
0809 09:32:33.311167 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:32:33.311171 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:32:33.311225 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:32:33.311316 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 09:32:33.312123 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:33.312960 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:33.314265 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:33.314973 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:33.315107 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:33.315190 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0809 09:32:33.315504 (+ 314us) server_negotiation.cc:299] Negotiation successful
0809 09:32:33.315583 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260809 09:32:33.315670 23881 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "daa39432dbc14ab198ef697006127abd" candidate_uuid: "768749257e7f437693e5f9a234a3db0c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff6decf4b31348898adbb75faaab9faf" is_pre_election: true
I20260809 09:32:33.316049 23881 raft_consensus.cc:2454] T daa39432dbc14ab198ef697006127abd P ff6decf4b31348898adbb75faaab9faf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 768749257e7f437693e5f9a234a3db0c in term 0.
I20260809 09:32:33.316458 24095 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "daa39432dbc14ab198ef697006127abd" candidate_uuid: "768749257e7f437693e5f9a234a3db0c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed89bfa60b114ee6a11787f6913bf1be" is_pre_election: true
I20260809 09:32:33.316679 24211 leader_election.cc:302] T daa39432dbc14ab198ef697006127abd P 768749257e7f437693e5f9a234a3db0c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 768749257e7f437693e5f9a234a3db0c, ff6decf4b31348898adbb75faaab9faf; no voters:
I20260809 09:32:33.316749 24095 raft_consensus.cc:2454] T daa39432dbc14ab198ef697006127abd P ed89bfa60b114ee6a11787f6913bf1be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 768749257e7f437693e5f9a234a3db0c in term 0.
I20260809 09:32:33.316998 24423 raft_consensus.cc:2781] T daa39432dbc14ab198ef697006127abd P 768749257e7f437693e5f9a234a3db0c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 09:32:33.317101 24423 raft_consensus.cc:482] T daa39432dbc14ab198ef697006127abd P 768749257e7f437693e5f9a234a3db0c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 09:32:33.317173 24423 raft_consensus.cc:3037] T daa39432dbc14ab198ef697006127abd P 768749257e7f437693e5f9a234a3db0c [term 0 FOLLOWER]: Advancing to term 1
I20260809 09:32:33.318334 24423 raft_consensus.cc:504] T daa39432dbc14ab198ef697006127abd P 768749257e7f437693e5f9a234a3db0c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff6decf4b31348898adbb75faaab9faf" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 37361 } } peers { permanent_uuid: "ed89bfa60b114ee6a11787f6913bf1be" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 37329 } } peers { permanent_uuid: "768749257e7f437693e5f9a234a3db0c" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 38953 } }
I20260809 09:32:33.318780 24423 leader_election.cc:288] T daa39432dbc14ab198ef697006127abd P 768749257e7f437693e5f9a234a3db0c [CANDIDATE]: Term 1 election: Requested vote from peers ff6decf4b31348898adbb75faaab9faf (127.20.125.129:37361), ed89bfa60b114ee6a11787f6913bf1be (127.20.125.130:37329)
I20260809 09:32:33.319151 23881 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "daa39432dbc14ab198ef697006127abd" candidate_uuid: "768749257e7f437693e5f9a234a3db0c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff6decf4b31348898adbb75faaab9faf"
I20260809 09:32:33.319308 23881 raft_consensus.cc:3037] T daa39432dbc14ab198ef697006127abd P ff6decf4b31348898adbb75faaab9faf [term 0 FOLLOWER]: Advancing to term 1
I20260809 09:32:33.319326 24095 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "daa39432dbc14ab198ef697006127abd" candidate_uuid: "768749257e7f437693e5f9a234a3db0c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed89bfa60b114ee6a11787f6913bf1be"
I20260809 09:32:33.319466 24095 raft_consensus.cc:3037] T daa39432dbc14ab198ef697006127abd P ed89bfa60b114ee6a11787f6913bf1be [term 0 FOLLOWER]: Advancing to term 1
I20260809 09:32:33.320497 23881 raft_consensus.cc:2454] T daa39432dbc14ab198ef697006127abd P ff6decf4b31348898adbb75faaab9faf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 768749257e7f437693e5f9a234a3db0c in term 1.
I20260809 09:32:33.320539 24095 raft_consensus.cc:2454] T daa39432dbc14ab198ef697006127abd P ed89bfa60b114ee6a11787f6913bf1be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 768749257e7f437693e5f9a234a3db0c in term 1.
I20260809 09:32:33.320849 24211 leader_election.cc:302] T daa39432dbc14ab198ef697006127abd P 768749257e7f437693e5f9a234a3db0c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 768749257e7f437693e5f9a234a3db0c, ed89bfa60b114ee6a11787f6913bf1be; no voters:
I20260809 09:32:33.321095 24423 raft_consensus.cc:2781] T daa39432dbc14ab198ef697006127abd P 768749257e7f437693e5f9a234a3db0c [term 1 FOLLOWER]: Leader election won for term 1
I20260809 09:32:33.321334 24423 raft_consensus.cc:686] T daa39432dbc14ab198ef697006127abd P 768749257e7f437693e5f9a234a3db0c [term 1 LEADER]: Becoming Leader. State: Replica: 768749257e7f437693e5f9a234a3db0c, State: Running, Role: LEADER
I20260809 09:32:33.321718 24423 consensus_queue.cc:237] T daa39432dbc14ab198ef697006127abd P 768749257e7f437693e5f9a234a3db0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff6decf4b31348898adbb75faaab9faf" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 37361 } } peers { permanent_uuid: "ed89bfa60b114ee6a11787f6913bf1be" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 37329 } } peers { permanent_uuid: "768749257e7f437693e5f9a234a3db0c" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 38953 } }
I20260809 09:32:33.325604 23643 catalog_manager.cc:5495] T daa39432dbc14ab198ef697006127abd P 768749257e7f437693e5f9a234a3db0c reported cstate change: term changed from 0 to 1, leader changed from <none> to 768749257e7f437693e5f9a234a3db0c (127.20.125.131). New cstate: current_term: 1 leader_uuid: "768749257e7f437693e5f9a234a3db0c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff6decf4b31348898adbb75faaab9faf" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 37361 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ed89bfa60b114ee6a11787f6913bf1be" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 37329 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "768749257e7f437693e5f9a234a3db0c" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 38953 } health_report { overall_health: HEALTHY } } }
Aug 09 09:32:33 dist-test-slave-7kzw krb5kdc[23584](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786267952, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.190@KRBTEST.COM for kudu/127.20.125.190@KRBTEST.COM
I20260809 09:32:33.377318 23725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:33.368516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60210 (local address 127.20.125.190:41195)
0809 09:32:33.368706 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0809 09:32:33.368712 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 09:32:33.368936 (+ 224us) server_negotiation.cc:407] Connection header received
0809 09:32:33.369096 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:32:33.369100 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:32:33.369144 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:32:33.369240 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:32:33.369935 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:33.370455 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:33.371309 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:33.371520 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:33.371627 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:33.374129 (+ 2502us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:32:33.374132 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:32:33.374134 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:32:33.374156 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:32:33.375558 (+ 1402us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:32:33.376037 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:32:33.376042 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:32:33.376044 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:32:33.376122 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:32:33.376409 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:32:33.376414 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:32:33.376416 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:32:33.376585 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:32:33.376693 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0809 09:32:33.376986 (+ 293us) server_negotiation.cc:299] Negotiation successful
0809 09:32:33.377153 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":65,"server-negotiator.queue_time_us":53}
I20260809 09:32:33.377319 24446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:33.368443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:41195 (local address 127.0.0.1:60210)
0809 09:32:33.368836 (+ 393us) negotiation.cc:107] Waiting for socket to connect
0809 09:32:33.368849 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 09:32:33.368996 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:32:33.369260 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:32:33.369263 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:32:33.369715 (+ 452us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:32:33.369811 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:32:33.369817 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:32:33.370611 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:32:33.370616 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:32:33.371178 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:32:33.371186 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:32:33.371634 (+ 448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:32:33.371637 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:32:33.371758 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:33.372355 (+ 597us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 09:32:33.372368 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:32:33.373985 (+ 1617us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 09:32:33.375679 (+ 1694us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:32:33.375696 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:32:33.375703 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:32:33.375937 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:32:33.376232 (+ 295us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:32:33.376235 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:32:33.376237 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:32:33.376321 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:32:33.376701 (+ 380us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 09:32:33.376710 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 09:32:33.376822 (+ 112us) client_negotiation.cc:769] Sending connection context
0809 09:32:33.376939 (+ 117us) client_negotiation.cc:240] Negotiation successful
0809 09:32:33.377095 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":316,"thread_start_us":127,"threads_started":1}
I20260809 09:32:33.381685 23643 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.20.125.190@KRBTEST.COM'} at 127.0.0.1:60210:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260809 09:32:33.382436 23643 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 09:32:33.387904 24067 tablet_service.cc:1467] Processing CreateTablet for tablet e55fae5b09c14429abcb1c526321471a (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=bcc6d58b87bf42fea2c00f5ef64f7992]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 09:32:33.387857 23851 tablet_service.cc:1467] Processing CreateTablet for tablet e55fae5b09c14429abcb1c526321471a (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=bcc6d58b87bf42fea2c00f5ef64f7992]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 09:32:33.387856 24288 tablet_service.cc:1467] Processing CreateTablet for tablet e55fae5b09c14429abcb1c526321471a (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=bcc6d58b87bf42fea2c00f5ef64f7992]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 09:32:33.388584 23851 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e55fae5b09c14429abcb1c526321471a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 09:32:33.388597 24288 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e55fae5b09c14429abcb1c526321471a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 09:32:33.388618 24067 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e55fae5b09c14429abcb1c526321471a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 09:32:33.391197 24418 tablet_bootstrap.cc:492] T e55fae5b09c14429abcb1c526321471a P ed89bfa60b114ee6a11787f6913bf1be: Bootstrap starting.
I20260809 09:32:33.391566 24414 tablet_bootstrap.cc:492] T e55fae5b09c14429abcb1c526321471a P ff6decf4b31348898adbb75faaab9faf: Bootstrap starting.
I20260809 09:32:33.391559 24417 tablet_bootstrap.cc:492] T e55fae5b09c14429abcb1c526321471a P 768749257e7f437693e5f9a234a3db0c: Bootstrap starting.
I20260809 09:32:33.392256 24418 tablet_bootstrap.cc:654] T e55fae5b09c14429abcb1c526321471a P ed89bfa60b114ee6a11787f6913bf1be: Neither blocks nor log segments found. Creating new log.
I20260809 09:32:33.392619 24417 tablet_bootstrap.cc:654] T e55fae5b09c14429abcb1c526321471a P 768749257e7f437693e5f9a234a3db0c: Neither blocks nor log segments found. Creating new log.
I20260809 09:32:33.392674 24414 tablet_bootstrap.cc:654] T e55fae5b09c14429abcb1c526321471a P ff6decf4b31348898adbb75faaab9faf: Neither blocks nor log segments found. Creating new log.
I20260809 09:32:33.393806 24418 tablet_bootstrap.cc:492] T e55fae5b09c14429abcb1c526321471a P ed89bfa60b114ee6a11787f6913bf1be: No bootstrap required, opened a new log
I20260809 09:32:33.393913 24418 ts_tablet_manager.cc:1397] T e55fae5b09c14429abcb1c526321471a P ed89bfa60b114ee6a11787f6913bf1be: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 09:32:33.394197 24414 tablet_bootstrap.cc:492] T e55fae5b09c14429abcb1c526321471a P ff6decf4b31348898adbb75faaab9faf: No bootstrap required, opened a new log
I20260809 09:32:33.394291 24414 ts_tablet_manager.cc:1397] T e55fae5b09c14429abcb1c526321471a P ff6decf4b31348898adbb75faaab9faf: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 09:32:33.394526 24418 raft_consensus.cc:348] T e55fae5b09c14429abcb1c526321471a P ed89bfa60b114ee6a11787f6913bf1be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff6decf4b31348898adbb75faaab9faf" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 37361 } } peers { permanent_uuid: "ed89bfa60b114ee6a11787f6913bf1be" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 37329 } } peers { permanent_uuid: "768749257e7f437693e5f9a234a3db0c" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 38953 } }
I20260809 09:32:33.394660 24418 raft_consensus.cc:374] T e55fae5b09c14429abcb1c526321471a P ed89bfa60b114ee6a11787f6913bf1be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 09:32:33.394704 24418 raft_consensus.cc:729] T e55fae5b09c14429abcb1c526321471a P ed89bfa60b114ee6a11787f6913bf1be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed89bfa60b114ee6a11787f6913bf1be, State: Initialized, Role: FOLLOWER
I20260809 09:32:33.394809 24414 raft_consensus.cc:348] T e55fae5b09c14429abcb1c526321471a P ff6decf4b31348898adbb75faaab9faf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff6decf4b31348898adbb75faaab9faf" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 37361 } } peers { permanent_uuid: "ed89bfa60b114ee6a11787f6913bf1be" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 37329 } } peers { permanent_uuid: "768749257e7f437693e5f9a234a3db0c" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 38953 } }
I20260809 09:32:33.394845 24418 consensus_queue.cc:260] T e55fae5b09c14429abcb1c526321471a P ed89bfa60b114ee6a11787f6913bf1be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff6decf4b31348898adbb75faaab9faf" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 37361 } } peers { permanent_uuid: "ed89bfa60b114ee6a11787f6913bf1be" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 37329 } } peers { permanent_uuid: "768749257e7f437693e5f9a234a3db0c" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 38953 } }
I20260809 09:32:33.394939 24414 raft_consensus.cc:374] T e55fae5b09c14429abcb1c526321471a P ff6decf4b31348898adbb75faaab9faf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 09:32:33.394979 24414 raft_consensus.cc:729] T e55fae5b09c14429abcb1c526321471a P ff6decf4b31348898adbb75faaab9faf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff6decf4b31348898adbb75faaab9faf, State: Initialized, Role: FOLLOWER
I20260809 09:32:33.395141 24418 ts_tablet_manager.cc:1428] T e55fae5b09c14429abcb1c526321471a P ed89bfa60b114ee6a11787f6913bf1be: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260809 09:32:33.395112 24414 consensus_queue.cc:260] T e55fae5b09c14429abcb1c526321471a P ff6decf4b31348898adbb75faaab9faf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff6decf4b31348898adbb75faaab9faf" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 37361 } } peers { permanent_uuid: "ed89bfa60b114ee6a11787f6913bf1be" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 37329 } } peers { permanent_uuid: "768749257e7f437693e5f9a234a3db0c" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 38953 } }
I20260809 09:32:33.395254 24428 tablet_replica.cc:439] T e55fae5b09c14429abcb1c526321471a P ed89bfa60b114ee6a11787f6913bf1be: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff6decf4b31348898adbb75faaab9faf" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 37361 } } peers { permanent_uuid: "ed89bfa60b114ee6a11787f6913bf1be" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 37329 } } peers { permanent_uuid: "768749257e7f437693e5f9a234a3db0c" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 38953 } } }
I20260809 09:32:33.395399 24428 tablet_replica.cc:442] T e55fae5b09c14429abcb1c526321471a P ed89bfa60b114ee6a11787f6913bf1be: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 09:32:33.395399 24414 ts_tablet_manager.cc:1428] T e55fae5b09c14429abcb1c526321471a P ff6decf4b31348898adbb75faaab9faf: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260809 09:32:33.395618 24422 tablet_replica.cc:439] T e55fae5b09c14429abcb1c526321471a P ff6decf4b31348898adbb75faaab9faf: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff6decf4b31348898adbb75faaab9faf" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 37361 } } peers { permanent_uuid: "ed89bfa60b114ee6a11787f6913bf1be" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 37329 } } peers { permanent_uuid: "768749257e7f437693e5f9a234a3db0c" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 38953 } } }
I20260809 09:32:33.395772 24422 tablet_replica.cc:442] T e55fae5b09c14429abcb1c526321471a P ff6decf4b31348898adbb75faaab9faf: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 09:32:33.396059 24417 tablet_bootstrap.cc:492] T e55fae5b09c14429abcb1c526321471a P 768749257e7f437693e5f9a234a3db0c: No bootstrap required, opened a new log
I20260809 09:32:33.396152 24417 ts_tablet_manager.cc:1397] T e55fae5b09c14429abcb1c526321471a P 768749257e7f437693e5f9a234a3db0c: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260809 09:32:33.396664 24417 raft_consensus.cc:348] T e55fae5b09c14429abcb1c526321471a P 768749257e7f437693e5f9a234a3db0c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff6decf4b31348898adbb75faaab9faf" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 37361 } } peers { permanent_uuid: "ed89bfa60b114ee6a11787f6913bf1be" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 37329 } } peers { permanent_uuid: "768749257e7f437693e5f9a234a3db0c" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 38953 } }
I20260809 09:32:33.396804 24417 raft_consensus.cc:374] T e55fae5b09c14429abcb1c526321471a P 768749257e7f437693e5f9a234a3db0c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 09:32:33.396838 24417 raft_consensus.cc:729] T e55fae5b09c14429abcb1c526321471a P 768749257e7f437693e5f9a234a3db0c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 768749257e7f437693e5f9a234a3db0c, State: Initialized, Role: FOLLOWER
I20260809 09:32:33.396981 24417 consensus_queue.cc:260] T e55fae5b09c14429abcb1c526321471a P 768749257e7f437693e5f9a234a3db0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff6decf4b31348898adbb75faaab9faf" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 37361 } } peers { permanent_uuid: "ed89bfa60b114ee6a11787f6913bf1be" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 37329 } } peers { permanent_uuid: "768749257e7f437693e5f9a234a3db0c" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 38953 } }
I20260809 09:32:33.397259 24417 ts_tablet_manager.cc:1428] T e55fae5b09c14429abcb1c526321471a P 768749257e7f437693e5f9a234a3db0c: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260809 09:32:33.397845 24423 tablet_replica.cc:439] T e55fae5b09c14429abcb1c526321471a P 768749257e7f437693e5f9a234a3db0c: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff6decf4b31348898adbb75faaab9faf" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 37361 } } peers { permanent_uuid: "ed89bfa60b114ee6a11787f6913bf1be" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 37329 } } peers { permanent_uuid: "768749257e7f437693e5f9a234a3db0c" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 38953 } } }
I20260809 09:32:33.397985 24423 tablet_replica.cc:442] T e55fae5b09c14429abcb1c526321471a P 768749257e7f437693e5f9a234a3db0c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 09:32:33.428560 24422 raft_consensus.cc:482] T e55fae5b09c14429abcb1c526321471a P ff6decf4b31348898adbb75faaab9faf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 09:32:33.428727 24422 raft_consensus.cc:504] T e55fae5b09c14429abcb1c526321471a P ff6decf4b31348898adbb75faaab9faf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff6decf4b31348898adbb75faaab9faf" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 37361 } } peers { permanent_uuid: "ed89bfa60b114ee6a11787f6913bf1be" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 37329 } } peers { permanent_uuid: "768749257e7f437693e5f9a234a3db0c" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 38953 } }
I20260809 09:32:33.430047 24422 leader_election.cc:288] T e55fae5b09c14429abcb1c526321471a P ff6decf4b31348898adbb75faaab9faf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ed89bfa60b114ee6a11787f6913bf1be (127.20.125.130:37329), 768749257e7f437693e5f9a234a3db0c (127.20.125.131:38953)
I20260809 09:32:33.435045 24413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:33.430718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.129:52941 (local address 127.20.125.131:38953)
0809 09:32:33.430854 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0809 09:32:33.430859 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 09:32:33.431022 (+ 163us) server_negotiation.cc:407] Connection header received
0809 09:32:33.431196 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:32:33.431201 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:32:33.431279 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:32:33.431375 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 09:32:33.431731 (+ 356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:33.432268 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:33.433644 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:33.434144 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:33.434285 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:33.434341 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0809 09:32:33.434790 (+ 449us) server_negotiation.cc:299] Negotiation successful
0809 09:32:33.434892 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 09:32:33.435072 24452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:33.430327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.131:38953 (local address 127.20.125.129:52941)
0809 09:32:33.430921 (+ 594us) negotiation.cc:107] Waiting for socket to connect
0809 09:32:33.430934 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 09:32:33.431070 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:32:33.431399 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:32:33.431402 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:32:33.431428 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 09:32:33.431570 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:32:33.431576 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:32:33.432387 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:32:33.432389 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:32:33.433476 (+ 1087us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:32:33.433488 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:32:33.434331 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:32:33.434335 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:32:33.434468 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:33.434562 (+ 94us) client_negotiation.cc:769] Sending connection context
0809 09:32:33.434771 (+ 209us) client_negotiation.cc:240] Negotiation successful
0809 09:32:33.434853 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":509,"thread_start_us":374,"threads_started":1}
I20260809 09:32:33.435192 24412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:33.430437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.129:58309 (local address 127.20.125.130:37329)
0809 09:32:33.430601 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0809 09:32:33.430606 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 09:32:33.431263 (+ 657us) server_negotiation.cc:407] Connection header received
0809 09:32:33.431455 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:32:33.431462 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:32:33.431534 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:32:33.431583 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 09:32:33.432380 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:33.432873 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:33.434098 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:33.434637 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:33.434788 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:33.434833 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0809 09:32:33.434995 (+ 162us) server_negotiation.cc:299] Negotiation successful
0809 09:32:33.435074 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260809 09:32:33.435207 24451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:33.430327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.130:37329 (local address 127.20.125.129:58309)
0809 09:32:33.431050 (+ 723us) negotiation.cc:107] Waiting for socket to connect
0809 09:32:33.431068 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 09:32:33.431292 (+ 224us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:32:33.431710 (+ 418us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:32:33.431712 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:32:33.431727 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 09:32:33.432225 (+ 498us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:32:33.432232 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:32:33.433002 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:32:33.433006 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:32:33.433951 (+ 945us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:32:33.433960 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:32:33.434768 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:32:33.434771 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:32:33.434862 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:33.434881 (+ 19us) client_negotiation.cc:769] Sending connection context
0809 09:32:33.434963 (+ 82us) client_negotiation.cc:240] Negotiation successful
0809 09:32:33.435015 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":631,"mutex_wait_us":196,"thread_start_us":145,"threads_started":1}
I20260809 09:32:33.435760 24317 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e55fae5b09c14429abcb1c526321471a" candidate_uuid: "ff6decf4b31348898adbb75faaab9faf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "768749257e7f437693e5f9a234a3db0c" is_pre_election: true
I20260809 09:32:33.435760 24095 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e55fae5b09c14429abcb1c526321471a" candidate_uuid: "ff6decf4b31348898adbb75faaab9faf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed89bfa60b114ee6a11787f6913bf1be" is_pre_election: true
I20260809 09:32:33.435994 24095 raft_consensus.cc:2454] T e55fae5b09c14429abcb1c526321471a P ed89bfa60b114ee6a11787f6913bf1be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff6decf4b31348898adbb75faaab9faf in term 0.
I20260809 09:32:33.436038 24317 raft_consensus.cc:2454] T e55fae5b09c14429abcb1c526321471a P 768749257e7f437693e5f9a234a3db0c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff6decf4b31348898adbb75faaab9faf in term 0.
I20260809 09:32:33.436609 23782 leader_election.cc:302] T e55fae5b09c14429abcb1c526321471a P ff6decf4b31348898adbb75faaab9faf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 768749257e7f437693e5f9a234a3db0c, ed89bfa60b114ee6a11787f6913bf1be, ff6decf4b31348898adbb75faaab9faf; no voters:
I20260809 09:32:33.436947 24422 raft_consensus.cc:2781] T e55fae5b09c14429abcb1c526321471a P ff6decf4b31348898adbb75faaab9faf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 09:32:33.437012 24422 raft_consensus.cc:482] T e55fae5b09c14429abcb1c526321471a P ff6decf4b31348898adbb75faaab9faf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 09:32:33.437040 24422 raft_consensus.cc:3037] T e55fae5b09c14429abcb1c526321471a P ff6decf4b31348898adbb75faaab9faf [term 0 FOLLOWER]: Advancing to term 1
I20260809 09:32:33.438057 24422 raft_consensus.cc:504] T e55fae5b09c14429abcb1c526321471a P ff6decf4b31348898adbb75faaab9faf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff6decf4b31348898adbb75faaab9faf" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 37361 } } peers { permanent_uuid: "ed89bfa60b114ee6a11787f6913bf1be" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 37329 } } peers { permanent_uuid: "768749257e7f437693e5f9a234a3db0c" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 38953 } }
I20260809 09:32:33.438453 24422 leader_election.cc:288] T e55fae5b09c14429abcb1c526321471a P ff6decf4b31348898adbb75faaab9faf [CANDIDATE]: Term 1 election: Requested vote from peers ed89bfa60b114ee6a11787f6913bf1be (127.20.125.130:37329), 768749257e7f437693e5f9a234a3db0c (127.20.125.131:38953)
I20260809 09:32:33.438836 24317 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e55fae5b09c14429abcb1c526321471a" candidate_uuid: "ff6decf4b31348898adbb75faaab9faf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "768749257e7f437693e5f9a234a3db0c"
I20260809 09:32:33.438969 24317 raft_consensus.cc:3037] T e55fae5b09c14429abcb1c526321471a P 768749257e7f437693e5f9a234a3db0c [term 0 FOLLOWER]: Advancing to term 1
I20260809 09:32:33.438963 24095 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e55fae5b09c14429abcb1c526321471a" candidate_uuid: "ff6decf4b31348898adbb75faaab9faf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed89bfa60b114ee6a11787f6913bf1be"
I20260809 09:32:33.439136 24095 raft_consensus.cc:3037] T e55fae5b09c14429abcb1c526321471a P ed89bfa60b114ee6a11787f6913bf1be [term 0 FOLLOWER]: Advancing to term 1
I20260809 09:32:33.440013 24317 raft_consensus.cc:2454] T e55fae5b09c14429abcb1c526321471a P 768749257e7f437693e5f9a234a3db0c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff6decf4b31348898adbb75faaab9faf in term 1.
I20260809 09:32:33.440215 24095 raft_consensus.cc:2454] T e55fae5b09c14429abcb1c526321471a P ed89bfa60b114ee6a11787f6913bf1be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff6decf4b31348898adbb75faaab9faf in term 1.
I20260809 09:32:33.440371 23779 leader_election.cc:302] T e55fae5b09c14429abcb1c526321471a P ff6decf4b31348898adbb75faaab9faf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 768749257e7f437693e5f9a234a3db0c, ff6decf4b31348898adbb75faaab9faf; no voters:
I20260809 09:32:33.440582 24422 raft_consensus.cc:2781] T e55fae5b09c14429abcb1c526321471a P ff6decf4b31348898adbb75faaab9faf [term 1 FOLLOWER]: Leader election won for term 1
I20260809 09:32:33.440816 24422 raft_consensus.cc:686] T e55fae5b09c14429abcb1c526321471a P ff6decf4b31348898adbb75faaab9faf [term 1 LEADER]: Becoming Leader. State: Replica: ff6decf4b31348898adbb75faaab9faf, State: Running, Role: LEADER
I20260809 09:32:33.441222 24422 consensus_queue.cc:237] T e55fae5b09c14429abcb1c526321471a P ff6decf4b31348898adbb75faaab9faf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff6decf4b31348898adbb75faaab9faf" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 37361 } } peers { permanent_uuid: "ed89bfa60b114ee6a11787f6913bf1be" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 37329 } } peers { permanent_uuid: "768749257e7f437693e5f9a234a3db0c" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 38953 } }
I20260809 09:32:33.443487 24453 tablet_replica.cc:439] T e55fae5b09c14429abcb1c526321471a P ff6decf4b31348898adbb75faaab9faf: TxnStatusTablet state changed. Reason: New leader ff6decf4b31348898adbb75faaab9faf. Latest consensus state: current_term: 1 leader_uuid: "ff6decf4b31348898adbb75faaab9faf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff6decf4b31348898adbb75faaab9faf" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 37361 } } peers { permanent_uuid: "ed89bfa60b114ee6a11787f6913bf1be" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 37329 } } peers { permanent_uuid: "768749257e7f437693e5f9a234a3db0c" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 38953 } } }
I20260809 09:32:33.443616 24453 tablet_replica.cc:442] T e55fae5b09c14429abcb1c526321471a P ff6decf4b31348898adbb75faaab9faf: This TxnStatusTablet replica's current role is: LEADER
I20260809 09:32:33.443950 24457 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260809 09:32:33.444172 24457 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260809 09:32:33.445043 23644 catalog_manager.cc:5495] T e55fae5b09c14429abcb1c526321471a P ff6decf4b31348898adbb75faaab9faf reported cstate change: term changed from 0 to 1, leader changed from <none> to ff6decf4b31348898adbb75faaab9faf (127.20.125.129). New cstate: current_term: 1 leader_uuid: "ff6decf4b31348898adbb75faaab9faf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff6decf4b31348898adbb75faaab9faf" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 37361 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ed89bfa60b114ee6a11787f6913bf1be" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 37329 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "768749257e7f437693e5f9a234a3db0c" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 38953 } health_report { overall_health: UNKNOWN } } }
I20260809 09:32:33.465312 24403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:33.461496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48190 (local address 127.20.125.129:37361)
0809 09:32:33.461655 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0809 09:32:33.461660 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 09:32:33.461678 (+ 18us) server_negotiation.cc:407] Connection header received
0809 09:32:33.461842 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:32:33.461845 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:32:33.461904 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:32:33.462022 (+ 118us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 09:32:33.462293 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:33.462845 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:33.463772 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:33.463991 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:33.464101 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:33.464443 (+ 342us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 09:32:33.464870 (+ 427us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 09:32:33.465000 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0809 09:32:33.465072 (+ 72us) server_negotiation.cc:299] Negotiation successful
0809 09:32:33.465130 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260809 09:32:33.465284 24446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:33.461355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.129:37361 (local address 127.0.0.1:48190)
0809 09:32:33.461515 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0809 09:32:33.461533 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 09:32:33.461659 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:32:33.462039 (+ 380us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:32:33.462042 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:32:33.462051 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 09:32:33.462173 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:32:33.462184 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:32:33.462965 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:32:33.462969 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:32:33.463633 (+ 664us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:32:33.463642 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:32:33.464130 (+ 488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:32:33.464134 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:32:33.464240 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:33.464283 (+ 43us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 09:32:33.464975 (+ 692us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 09:32:33.464992 (+ 17us) client_negotiation.cc:769] Sending connection context
0809 09:32:33.465064 (+ 72us) client_negotiation.cc:240] Negotiation successful
0809 09:32:33.465166 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52}
I20260809 09:32:33.470948 24317 raft_consensus.cc:1261] T e55fae5b09c14429abcb1c526321471a P 768749257e7f437693e5f9a234a3db0c [term 1 FOLLOWER]: Refusing update from remote peer ff6decf4b31348898adbb75faaab9faf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 09:32:33.471212 24095 raft_consensus.cc:1261] T e55fae5b09c14429abcb1c526321471a P ed89bfa60b114ee6a11787f6913bf1be [term 1 FOLLOWER]: Refusing update from remote peer ff6decf4b31348898adbb75faaab9faf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 09:32:33.471853 24453 consensus_queue.cc:1035] T e55fae5b09c14429abcb1c526321471a P ff6decf4b31348898adbb75faaab9faf [LEADER]: Connected to new peer: Peer: permanent_uuid: "768749257e7f437693e5f9a234a3db0c" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 38953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 09:32:33.472083 24422 consensus_queue.cc:1035] T e55fae5b09c14429abcb1c526321471a P ff6decf4b31348898adbb75faaab9faf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ed89bfa60b114ee6a11787f6913bf1be" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 37329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 09:32:33.475783 24423 tablet_replica.cc:439] T e55fae5b09c14429abcb1c526321471a P 768749257e7f437693e5f9a234a3db0c: TxnStatusTablet state changed. Reason: New leader ff6decf4b31348898adbb75faaab9faf. Latest consensus state: current_term: 1 leader_uuid: "ff6decf4b31348898adbb75faaab9faf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff6decf4b31348898adbb75faaab9faf" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 37361 } } peers { permanent_uuid: "ed89bfa60b114ee6a11787f6913bf1be" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 37329 } } peers { permanent_uuid: "768749257e7f437693e5f9a234a3db0c" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 38953 } } }
I20260809 09:32:33.475924 24423 tablet_replica.cc:442] T e55fae5b09c14429abcb1c526321471a P 768749257e7f437693e5f9a234a3db0c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 09:32:33.476046 24428 tablet_replica.cc:439] T e55fae5b09c14429abcb1c526321471a P ed89bfa60b114ee6a11787f6913bf1be: TxnStatusTablet state changed. Reason: New leader ff6decf4b31348898adbb75faaab9faf. Latest consensus state: current_term: 1 leader_uuid: "ff6decf4b31348898adbb75faaab9faf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff6decf4b31348898adbb75faaab9faf" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 37361 } } peers { permanent_uuid: "ed89bfa60b114ee6a11787f6913bf1be" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 37329 } } peers { permanent_uuid: "768749257e7f437693e5f9a234a3db0c" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 38953 } } }
I20260809 09:32:33.476161 24428 tablet_replica.cc:442] T e55fae5b09c14429abcb1c526321471a P ed89bfa60b114ee6a11787f6913bf1be: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 09:32:33.479607 24460 mvcc.cc:205] Tried to move back new op lower bound from 7316553537409916928 to 7316553537298579456. Current Snapshot: MvccSnapshot[applied={T|T < 7316553537409916928}]
I20260809 09:32:33.479760 24423 tablet_replica.cc:439] T e55fae5b09c14429abcb1c526321471a P 768749257e7f437693e5f9a234a3db0c: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ff6decf4b31348898adbb75faaab9faf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff6decf4b31348898adbb75faaab9faf" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 37361 } } peers { permanent_uuid: "ed89bfa60b114ee6a11787f6913bf1be" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 37329 } } peers { permanent_uuid: "768749257e7f437693e5f9a234a3db0c" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 38953 } } }
I20260809 09:32:33.479875 24423 tablet_replica.cc:442] T e55fae5b09c14429abcb1c526321471a P 768749257e7f437693e5f9a234a3db0c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 09:32:33.480020 24422 tablet_replica.cc:439] T e55fae5b09c14429abcb1c526321471a P ff6decf4b31348898adbb75faaab9faf: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ff6decf4b31348898adbb75faaab9faf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff6decf4b31348898adbb75faaab9faf" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 37361 } } peers { permanent_uuid: "ed89bfa60b114ee6a11787f6913bf1be" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 37329 } } peers { permanent_uuid: "768749257e7f437693e5f9a234a3db0c" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 38953 } } }
I20260809 09:32:33.480141 24422 tablet_replica.cc:442] T e55fae5b09c14429abcb1c526321471a P ff6decf4b31348898adbb75faaab9faf: This TxnStatusTablet replica's current role is: LEADER
I20260809 09:32:33.480321 24422 tablet_replica.cc:439] T e55fae5b09c14429abcb1c526321471a P ff6decf4b31348898adbb75faaab9faf: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ff6decf4b31348898adbb75faaab9faf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff6decf4b31348898adbb75faaab9faf" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 37361 } } peers { permanent_uuid: "ed89bfa60b114ee6a11787f6913bf1be" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 37329 } } peers { permanent_uuid: "768749257e7f437693e5f9a234a3db0c" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 38953 } } }
I20260809 09:32:33.480403 24422 tablet_replica.cc:442] T e55fae5b09c14429abcb1c526321471a P ff6decf4b31348898adbb75faaab9faf: This TxnStatusTablet replica's current role is: LEADER
I20260809 09:32:33.481523 24428 tablet_replica.cc:439] T e55fae5b09c14429abcb1c526321471a P ed89bfa60b114ee6a11787f6913bf1be: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ff6decf4b31348898adbb75faaab9faf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff6decf4b31348898adbb75faaab9faf" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 37361 } } peers { permanent_uuid: "ed89bfa60b114ee6a11787f6913bf1be" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 37329 } } peers { permanent_uuid: "768749257e7f437693e5f9a234a3db0c" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 38953 } } }
I20260809 09:32:33.481657 24428 tablet_replica.cc:442] T e55fae5b09c14429abcb1c526321471a P ed89bfa60b114ee6a11787f6913bf1be: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 09:32:33.481716 24463 mvcc.cc:205] Tried to move back new op lower bound from 7316553537409916928 to 7316553537298579456. Current Snapshot: MvccSnapshot[applied={T|T < 7316553537409916928}]
I20260809 09:32:33.484220 24462 mvcc.cc:205] Tried to move back new op lower bound from 7316553537409916928 to 7316553537298579456. Current Snapshot: MvccSnapshot[applied={T|T < 7316553537409916928}]
I20260809 09:32:33.493155 24413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:33.489397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57346 (local address 127.20.125.131:38953)
0809 09:32:33.489552 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0809 09:32:33.489556 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 09:32:33.489570 (+ 14us) server_negotiation.cc:407] Connection header received
0809 09:32:33.489642 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:32:33.489644 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:32:33.489699 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:32:33.489808 (+ 109us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 09:32:33.490077 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:33.490657 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:33.491601 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:33.491839 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:33.491982 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:33.492303 (+ 321us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 09:32:33.492753 (+ 450us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 09:32:33.492882 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0809 09:32:33.492947 (+ 65us) server_negotiation.cc:299] Negotiation successful
0809 09:32:33.493004 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260809 09:32:33.506321 24390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:33.502248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.129:37361 (local address 127.20.125.131:41663)
0809 09:32:33.502436 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0809 09:32:33.502453 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 09:32:33.502571 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:32:33.502929 (+ 358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:32:33.502933 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:32:33.502942 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 09:32:33.503069 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:32:33.503075 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:32:33.503892 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:32:33.503895 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:32:33.504840 (+ 945us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:32:33.504852 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:32:33.505367 (+ 515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:32:33.505371 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:32:33.505472 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:33.505526 (+ 54us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 09:32:33.505989 (+ 463us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 09:32:33.505996 (+ 7us) client_negotiation.cc:769] Sending connection context
0809 09:32:33.506069 (+ 73us) client_negotiation.cc:240] Negotiation successful
0809 09:32:33.506151 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":75}
I20260809 09:32:33.506441 24403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:33.502436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.131:41663 (local address 127.20.125.129:37361)
0809 09:32:33.502685 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0809 09:32:33.502690 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 09:32:33.502706 (+ 16us) server_negotiation.cc:407] Connection header received
0809 09:32:33.502765 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:32:33.502768 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:32:33.502823 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:32:33.502885 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 09:32:33.503216 (+ 331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:33.503771 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:33.504987 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:33.505232 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:33.505342 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:33.505737 (+ 395us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 09:32:33.505831 (+ 94us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 09:32:33.505946 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0809 09:32:33.506240 (+ 294us) server_negotiation.cc:299] Negotiation successful
0809 09:32:33.506299 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154}
I20260809 09:32:33.514792 23881 raft_consensus.cc:1261] T daa39432dbc14ab198ef697006127abd P ff6decf4b31348898adbb75faaab9faf [term 1 FOLLOWER]: Refusing update from remote peer 768749257e7f437693e5f9a234a3db0c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 09:32:33.514804 24095 raft_consensus.cc:1261] T daa39432dbc14ab198ef697006127abd P ed89bfa60b114ee6a11787f6913bf1be [term 1 FOLLOWER]: Refusing update from remote peer 768749257e7f437693e5f9a234a3db0c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 09:32:33.515410 24434 consensus_queue.cc:1035] T daa39432dbc14ab198ef697006127abd P 768749257e7f437693e5f9a234a3db0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ed89bfa60b114ee6a11787f6913bf1be" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 37329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 09:32:33.515614 24423 consensus_queue.cc:1035] T daa39432dbc14ab198ef697006127abd P 768749257e7f437693e5f9a234a3db0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff6decf4b31348898adbb75faaab9faf" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 37361 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 09:32:33.543740 24413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:33.539244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.129:49885 (local address 127.20.125.131:38953)
0809 09:32:33.539356 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0809 09:32:33.539361 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 09:32:33.539602 (+ 241us) server_negotiation.cc:407] Connection header received
0809 09:32:33.539940 (+ 338us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:32:33.539943 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:32:33.539993 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:32:33.540035 (+ 42us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 09:32:33.540406 (+ 371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:33.541188 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:33.542400 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:33.542784 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:33.542880 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:33.543275 (+ 395us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 09:32:33.543356 (+ 81us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 09:32:33.543395 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0809 09:32:33.543536 (+ 141us) server_negotiation.cc:299] Negotiation successful
0809 09:32:33.543588 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260809 09:32:33.544118 24489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:33.539138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.131:38953 (local address 127.20.125.129:49885)
0809 09:32:33.539506 (+ 368us) negotiation.cc:107] Waiting for socket to connect
0809 09:32:33.539521 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 09:32:33.539816 (+ 295us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:32:33.540155 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:32:33.540158 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:32:33.540177 (+ 19us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 09:32:33.540296 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:32:33.540302 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:32:33.541284 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:32:33.541288 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:32:33.542269 (+ 981us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:32:33.542280 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:32:33.542959 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:32:33.542962 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:32:33.543062 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:33.543114 (+ 52us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 09:32:33.543451 (+ 337us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 09:32:33.543455 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 09:32:33.543896 (+ 441us) client_negotiation.cc:240] Negotiation successful
0809 09:32:33.543980 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":112,"threads_started":1}
I20260809 09:32:33.570979 23644 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39006:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 09:32:33.571194 23644 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39006:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 09:32:33.574164 23644 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 22534bf98c554d6c8acdd00f8d2a9a29: Sending DeleteTablet for 3 replicas of tablet daa39432dbc14ab198ef697006127abd
I20260809 09:32:33.575105 24288 tablet_service.cc:1514] Processing DeleteTablet for tablet daa39432dbc14ab198ef697006127abd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 09:32:33 UTC) from {username='kudu', principal='kudu/127.20.125.190@KRBTEST.COM'} at 127.0.0.1:45356
I20260809 09:32:33.575373 24067 tablet_service.cc:1514] Processing DeleteTablet for tablet daa39432dbc14ab198ef697006127abd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 09:32:33 UTC) from {username='kudu', principal='kudu/127.20.125.190@KRBTEST.COM'} at 127.0.0.1:48072
I20260809 09:32:33.575855 24495 tablet_replica.cc:330] T daa39432dbc14ab198ef697006127abd P ed89bfa60b114ee6a11787f6913bf1be: stopping tablet replica
I20260809 09:32:33.575964 23851 tablet_service.cc:1514] Processing DeleteTablet for tablet daa39432dbc14ab198ef697006127abd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 09:32:33 UTC) from {username='kudu', principal='kudu/127.20.125.190@KRBTEST.COM'} at 127.0.0.1:35230
I20260809 09:32:33.576193 24495 raft_consensus.cc:2229] T daa39432dbc14ab198ef697006127abd P ed89bfa60b114ee6a11787f6913bf1be [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 09:32:33.576411 24495 raft_consensus.cc:2258] T daa39432dbc14ab198ef697006127abd P ed89bfa60b114ee6a11787f6913bf1be [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 09:32:33.577316 23644 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39006:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 09:32:33.577736 23644 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 09:32:33.578502 24495 ts_tablet_manager.cc:1905] T daa39432dbc14ab198ef697006127abd P ed89bfa60b114ee6a11787f6913bf1be: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 09:32:33.581859 24495 ts_tablet_manager.cc:1918] T daa39432dbc14ab198ef697006127abd P ed89bfa60b114ee6a11787f6913bf1be: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 09:32:33.582103 24495 log.cc:1196] T daa39432dbc14ab198ef697006127abd P ed89bfa60b114ee6a11787f6913bf1be: Deleting WAL directory at /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786267941944493-20982-0/minicluster-data/ts-1/wal/wals/daa39432dbc14ab198ef697006127abd
I20260809 09:32:33.582619 24495 ts_tablet_manager.cc:1939] T daa39432dbc14ab198ef697006127abd P ed89bfa60b114ee6a11787f6913bf1be: Deleting consensus metadata
I20260809 09:32:33.582651 24496 tablet_replica.cc:330] T daa39432dbc14ab198ef697006127abd P ff6decf4b31348898adbb75faaab9faf: stopping tablet replica
I20260809 09:32:33.582984 24496 raft_consensus.cc:2229] T daa39432dbc14ab198ef697006127abd P ff6decf4b31348898adbb75faaab9faf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 09:32:33.583174 24496 raft_consensus.cc:2258] T daa39432dbc14ab198ef697006127abd P ff6decf4b31348898adbb75faaab9faf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 09:32:33.583244 23851 tablet_service.cc:1467] Processing CreateTablet for tablet 8d47fee6639b4a57a0f4362cb2083b02 (DEFAULT_TABLE table=test-table [id=0d1150d44c15434996c8d26a74c66e28]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 09:32:33.583519 23851 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8d47fee6639b4a57a0f4362cb2083b02. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 09:32:33.583519 24067 tablet_service.cc:1467] Processing CreateTablet for tablet 8d47fee6639b4a57a0f4362cb2083b02 (DEFAULT_TABLE table=test-table [id=0d1150d44c15434996c8d26a74c66e28]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 09:32:33.583772 24067 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8d47fee6639b4a57a0f4362cb2083b02. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 09:32:33.584482 24496 ts_tablet_manager.cc:1905] T daa39432dbc14ab198ef697006127abd P ff6decf4b31348898adbb75faaab9faf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 09:32:33.586191 24414 tablet_bootstrap.cc:492] T 8d47fee6639b4a57a0f4362cb2083b02 P ff6decf4b31348898adbb75faaab9faf: Bootstrap starting.
I20260809 09:32:33.586262 24288 tablet_service.cc:1467] Processing CreateTablet for tablet 8d47fee6639b4a57a0f4362cb2083b02 (DEFAULT_TABLE table=test-table [id=0d1150d44c15434996c8d26a74c66e28]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 09:32:33.586548 24288 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8d47fee6639b4a57a0f4362cb2083b02. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 09:32:33.587234 24414 tablet_bootstrap.cc:654] T 8d47fee6639b4a57a0f4362cb2083b02 P ff6decf4b31348898adbb75faaab9faf: Neither blocks nor log segments found. Creating new log.
I20260809 09:32:33.588856 24496 ts_tablet_manager.cc:1918] T daa39432dbc14ab198ef697006127abd P ff6decf4b31348898adbb75faaab9faf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 09:32:33.589015 24417 tablet_bootstrap.cc:492] T 8d47fee6639b4a57a0f4362cb2083b02 P 768749257e7f437693e5f9a234a3db0c: Bootstrap starting.
I20260809 09:32:33.589123 24418 tablet_bootstrap.cc:492] T 8d47fee6639b4a57a0f4362cb2083b02 P ed89bfa60b114ee6a11787f6913bf1be: Bootstrap starting.
I20260809 09:32:33.589995 24417 tablet_bootstrap.cc:654] T 8d47fee6639b4a57a0f4362cb2083b02 P 768749257e7f437693e5f9a234a3db0c: Neither blocks nor log segments found. Creating new log.
I20260809 09:32:33.590013 24418 tablet_bootstrap.cc:654] T 8d47fee6639b4a57a0f4362cb2083b02 P ed89bfa60b114ee6a11787f6913bf1be: Neither blocks nor log segments found. Creating new log.
I20260809 09:32:33.590819 23626 catalog_manager.cc:4841] TS ed89bfa60b114ee6a11787f6913bf1be (127.20.125.130:37329): tablet daa39432dbc14ab198ef697006127abd (table test-table [id=856cdbf2690a418f94d612f9abac6c84]) successfully deleted
I20260809 09:32:33.591395 24417 tablet_bootstrap.cc:492] T 8d47fee6639b4a57a0f4362cb2083b02 P 768749257e7f437693e5f9a234a3db0c: No bootstrap required, opened a new log
I20260809 09:32:33.591478 24417 ts_tablet_manager.cc:1397] T 8d47fee6639b4a57a0f4362cb2083b02 P 768749257e7f437693e5f9a234a3db0c: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260809 09:32:33.591943 24417 raft_consensus.cc:348] T 8d47fee6639b4a57a0f4362cb2083b02 P 768749257e7f437693e5f9a234a3db0c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff6decf4b31348898adbb75faaab9faf" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 37361 } } peers { permanent_uuid: "ed89bfa60b114ee6a11787f6913bf1be" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 37329 } } peers { permanent_uuid: "768749257e7f437693e5f9a234a3db0c" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 38953 } }
I20260809 09:32:33.592060 24417 raft_consensus.cc:374] T 8d47fee6639b4a57a0f4362cb2083b02 P 768749257e7f437693e5f9a234a3db0c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 09:32:33.592090 24417 raft_consensus.cc:729] T 8d47fee6639b4a57a0f4362cb2083b02 P 768749257e7f437693e5f9a234a3db0c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 768749257e7f437693e5f9a234a3db0c, State: Initialized, Role: FOLLOWER
I20260809 09:32:33.592213 24417 consensus_queue.cc:260] T 8d47fee6639b4a57a0f4362cb2083b02 P 768749257e7f437693e5f9a234a3db0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff6decf4b31348898adbb75faaab9faf" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 37361 } } peers { permanent_uuid: "ed89bfa60b114ee6a11787f6913bf1be" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 37329 } } peers { permanent_uuid: "768749257e7f437693e5f9a234a3db0c" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 38953 } }
I20260809 09:32:33.592494 24417 ts_tablet_manager.cc:1428] T 8d47fee6639b4a57a0f4362cb2083b02 P 768749257e7f437693e5f9a234a3db0c: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260809 09:32:33.592552 24493 tablet_replica.cc:330] T daa39432dbc14ab198ef697006127abd P 768749257e7f437693e5f9a234a3db0c: stopping tablet replica
I20260809 09:32:33.590219 24414 tablet_bootstrap.cc:492] T 8d47fee6639b4a57a0f4362cb2083b02 P ff6decf4b31348898adbb75faaab9faf: No bootstrap required, opened a new log
I20260809 09:32:33.592801 24414 ts_tablet_manager.cc:1397] T 8d47fee6639b4a57a0f4362cb2083b02 P ff6decf4b31348898adbb75faaab9faf: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20260809 09:32:33.592871 24493 raft_consensus.cc:2229] T daa39432dbc14ab198ef697006127abd P 768749257e7f437693e5f9a234a3db0c [term 1 LEADER]: Raft consensus shutting down.
I20260809 09:32:33.593243 24493 raft_consensus.cc:2258] T daa39432dbc14ab198ef697006127abd P 768749257e7f437693e5f9a234a3db0c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 09:32:33.593287 24414 raft_consensus.cc:348] T 8d47fee6639b4a57a0f4362cb2083b02 P ff6decf4b31348898adbb75faaab9faf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff6decf4b31348898adbb75faaab9faf" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 37361 } } peers { permanent_uuid: "ed89bfa60b114ee6a11787f6913bf1be" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 37329 } } peers { permanent_uuid: "768749257e7f437693e5f9a234a3db0c" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 38953 } }
I20260809 09:32:33.593402 24414 raft_consensus.cc:374] T 8d47fee6639b4a57a0f4362cb2083b02 P ff6decf4b31348898adbb75faaab9faf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 09:32:33.593437 24414 raft_consensus.cc:729] T 8d47fee6639b4a57a0f4362cb2083b02 P ff6decf4b31348898adbb75faaab9faf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff6decf4b31348898adbb75faaab9faf, State: Initialized, Role: FOLLOWER
I20260809 09:32:33.593575 24414 consensus_queue.cc:260] T 8d47fee6639b4a57a0f4362cb2083b02 P ff6decf4b31348898adbb75faaab9faf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff6decf4b31348898adbb75faaab9faf" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 37361 } } peers { permanent_uuid: "ed89bfa60b114ee6a11787f6913bf1be" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 37329 } } peers { permanent_uuid: "768749257e7f437693e5f9a234a3db0c" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 38953 } }
I20260809 09:32:33.593859 24414 ts_tablet_manager.cc:1428] T 8d47fee6639b4a57a0f4362cb2083b02 P ff6decf4b31348898adbb75faaab9faf: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260809 09:32:33.594532 24496 log.cc:1196] T daa39432dbc14ab198ef697006127abd P ff6decf4b31348898adbb75faaab9faf: Deleting WAL directory at /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786267941944493-20982-0/minicluster-data/ts-0/wal/wals/daa39432dbc14ab198ef697006127abd
I20260809 09:32:33.595037 24496 ts_tablet_manager.cc:1939] T daa39432dbc14ab198ef697006127abd P ff6decf4b31348898adbb75faaab9faf: Deleting consensus metadata
I20260809 09:32:33.595469 24434 raft_consensus.cc:482] T 8d47fee6639b4a57a0f4362cb2083b02 P 768749257e7f437693e5f9a234a3db0c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 09:32:33.595573 24434 raft_consensus.cc:504] T 8d47fee6639b4a57a0f4362cb2083b02 P 768749257e7f437693e5f9a234a3db0c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff6decf4b31348898adbb75faaab9faf" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 37361 } } peers { permanent_uuid: "ed89bfa60b114ee6a11787f6913bf1be" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 37329 } } peers { permanent_uuid: "768749257e7f437693e5f9a234a3db0c" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 38953 } }
I20260809 09:32:33.595965 24434 leader_election.cc:288] T 8d47fee6639b4a57a0f4362cb2083b02 P 768749257e7f437693e5f9a234a3db0c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ff6decf4b31348898adbb75faaab9faf (127.20.125.129:37361), ed89bfa60b114ee6a11787f6913bf1be (127.20.125.130:37329)
I20260809 09:32:33.596486 24095 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d47fee6639b4a57a0f4362cb2083b02" candidate_uuid: "768749257e7f437693e5f9a234a3db0c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed89bfa60b114ee6a11787f6913bf1be" is_pre_election: true
I20260809 09:32:33.596606 23626 catalog_manager.cc:4841] TS ff6decf4b31348898adbb75faaab9faf (127.20.125.129:37361): tablet daa39432dbc14ab198ef697006127abd (table test-table [id=856cdbf2690a418f94d612f9abac6c84]) successfully deleted
I20260809 09:32:33.597087 23880 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d47fee6639b4a57a0f4362cb2083b02" candidate_uuid: "768749257e7f437693e5f9a234a3db0c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff6decf4b31348898adbb75faaab9faf" is_pre_election: true
I20260809 09:32:33.597251 23880 raft_consensus.cc:2454] T 8d47fee6639b4a57a0f4362cb2083b02 P ff6decf4b31348898adbb75faaab9faf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 768749257e7f437693e5f9a234a3db0c in term 0.
W20260809 09:32:33.597632 24211 leader_election.cc:341] T 8d47fee6639b4a57a0f4362cb2083b02 P 768749257e7f437693e5f9a234a3db0c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ed89bfa60b114ee6a11787f6913bf1be (127.20.125.130:37329): Illegal state: must be running to vote when last-logged opid is not known
I20260809 09:32:33.598021 24211 leader_election.cc:302] T 8d47fee6639b4a57a0f4362cb2083b02 P 768749257e7f437693e5f9a234a3db0c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 768749257e7f437693e5f9a234a3db0c, ff6decf4b31348898adbb75faaab9faf; no voters: ed89bfa60b114ee6a11787f6913bf1be
I20260809 09:32:33.598348 24434 raft_consensus.cc:2781] T 8d47fee6639b4a57a0f4362cb2083b02 P 768749257e7f437693e5f9a234a3db0c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 09:32:33.598416 24434 raft_consensus.cc:482] T 8d47fee6639b4a57a0f4362cb2083b02 P 768749257e7f437693e5f9a234a3db0c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 09:32:33.598481 24434 raft_consensus.cc:3037] T 8d47fee6639b4a57a0f4362cb2083b02 P 768749257e7f437693e5f9a234a3db0c [term 0 FOLLOWER]: Advancing to term 1
I20260809 09:32:33.599422 24493 ts_tablet_manager.cc:1905] T daa39432dbc14ab198ef697006127abd P 768749257e7f437693e5f9a234a3db0c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 09:32:33.599701 24434 raft_consensus.cc:504] T 8d47fee6639b4a57a0f4362cb2083b02 P 768749257e7f437693e5f9a234a3db0c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff6decf4b31348898adbb75faaab9faf" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 37361 } } peers { permanent_uuid: "ed89bfa60b114ee6a11787f6913bf1be" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 37329 } } peers { permanent_uuid: "768749257e7f437693e5f9a234a3db0c" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 38953 } }
I20260809 09:32:33.600051 24434 leader_election.cc:288] T 8d47fee6639b4a57a0f4362cb2083b02 P 768749257e7f437693e5f9a234a3db0c [CANDIDATE]: Term 1 election: Requested vote from peers ff6decf4b31348898adbb75faaab9faf (127.20.125.129:37361), ed89bfa60b114ee6a11787f6913bf1be (127.20.125.130:37329)
I20260809 09:32:33.600715 23880 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d47fee6639b4a57a0f4362cb2083b02" candidate_uuid: "768749257e7f437693e5f9a234a3db0c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff6decf4b31348898adbb75faaab9faf"
I20260809 09:32:33.600847 23880 raft_consensus.cc:3037] T 8d47fee6639b4a57a0f4362cb2083b02 P ff6decf4b31348898adbb75faaab9faf [term 0 FOLLOWER]: Advancing to term 1
I20260809 09:32:33.602041 23880 raft_consensus.cc:2454] T 8d47fee6639b4a57a0f4362cb2083b02 P ff6decf4b31348898adbb75faaab9faf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 768749257e7f437693e5f9a234a3db0c in term 1.
I20260809 09:32:33.602408 24095 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d47fee6639b4a57a0f4362cb2083b02" candidate_uuid: "768749257e7f437693e5f9a234a3db0c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed89bfa60b114ee6a11787f6913bf1be"
W20260809 09:32:33.602820 24211 leader_election.cc:341] T 8d47fee6639b4a57a0f4362cb2083b02 P 768749257e7f437693e5f9a234a3db0c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ed89bfa60b114ee6a11787f6913bf1be (127.20.125.130:37329): Illegal state: must be running to vote when last-logged opid is not known
I20260809 09:32:33.602949 24211 leader_election.cc:302] T 8d47fee6639b4a57a0f4362cb2083b02 P 768749257e7f437693e5f9a234a3db0c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 768749257e7f437693e5f9a234a3db0c, ff6decf4b31348898adbb75faaab9faf; no voters: ed89bfa60b114ee6a11787f6913bf1be
I20260809 09:32:33.603173 24434 raft_consensus.cc:2781] T 8d47fee6639b4a57a0f4362cb2083b02 P 768749257e7f437693e5f9a234a3db0c [term 1 FOLLOWER]: Leader election won for term 1
I20260809 09:32:33.603291 24434 raft_consensus.cc:686] T 8d47fee6639b4a57a0f4362cb2083b02 P 768749257e7f437693e5f9a234a3db0c [term 1 LEADER]: Becoming Leader. State: Replica: 768749257e7f437693e5f9a234a3db0c, State: Running, Role: LEADER
I20260809 09:32:33.603377 24493 ts_tablet_manager.cc:1918] T daa39432dbc14ab198ef697006127abd P 768749257e7f437693e5f9a234a3db0c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 09:32:33.603473 24493 log.cc:1196] T daa39432dbc14ab198ef697006127abd P 768749257e7f437693e5f9a234a3db0c: Deleting WAL directory at /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786267941944493-20982-0/minicluster-data/ts-2/wal/wals/daa39432dbc14ab198ef697006127abd
I20260809 09:32:33.603436 24434 consensus_queue.cc:237] T 8d47fee6639b4a57a0f4362cb2083b02 P 768749257e7f437693e5f9a234a3db0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff6decf4b31348898adbb75faaab9faf" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 37361 } } peers { permanent_uuid: "ed89bfa60b114ee6a11787f6913bf1be" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 37329 } } peers { permanent_uuid: "768749257e7f437693e5f9a234a3db0c" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 38953 } }
I20260809 09:32:33.605325 23644 catalog_manager.cc:5495] T 8d47fee6639b4a57a0f4362cb2083b02 P 768749257e7f437693e5f9a234a3db0c reported cstate change: term changed from 0 to 1, leader changed from <none> to 768749257e7f437693e5f9a234a3db0c (127.20.125.131). New cstate: current_term: 1 leader_uuid: "768749257e7f437693e5f9a234a3db0c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff6decf4b31348898adbb75faaab9faf" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 37361 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ed89bfa60b114ee6a11787f6913bf1be" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 37329 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "768749257e7f437693e5f9a234a3db0c" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 38953 } health_report { overall_health: HEALTHY } } }
I20260809 09:32:33.606321 24418 tablet_bootstrap.cc:492] T 8d47fee6639b4a57a0f4362cb2083b02 P ed89bfa60b114ee6a11787f6913bf1be: No bootstrap required, opened a new log
I20260809 09:32:33.606433 24418 ts_tablet_manager.cc:1397] T 8d47fee6639b4a57a0f4362cb2083b02 P ed89bfa60b114ee6a11787f6913bf1be: Time spent bootstrapping tablet: real 0.017s user 0.002s sys 0.000s
I20260809 09:32:33.604703 24493 ts_tablet_manager.cc:1939] T daa39432dbc14ab198ef697006127abd P 768749257e7f437693e5f9a234a3db0c: Deleting consensus metadata
I20260809 09:32:33.607019 24418 raft_consensus.cc:348] T 8d47fee6639b4a57a0f4362cb2083b02 P ed89bfa60b114ee6a11787f6913bf1be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff6decf4b31348898adbb75faaab9faf" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 37361 } } peers { permanent_uuid: "ed89bfa60b114ee6a11787f6913bf1be" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 37329 } } peers { permanent_uuid: "768749257e7f437693e5f9a234a3db0c" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 38953 } }
I20260809 09:32:33.607146 24418 raft_consensus.cc:374] T 8d47fee6639b4a57a0f4362cb2083b02 P ed89bfa60b114ee6a11787f6913bf1be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 09:32:33.607199 24418 raft_consensus.cc:729] T 8d47fee6639b4a57a0f4362cb2083b02 P ed89bfa60b114ee6a11787f6913bf1be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed89bfa60b114ee6a11787f6913bf1be, State: Initialized, Role: FOLLOWER
I20260809 09:32:33.607373 24418 consensus_queue.cc:260] T 8d47fee6639b4a57a0f4362cb2083b02 P ed89bfa60b114ee6a11787f6913bf1be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff6decf4b31348898adbb75faaab9faf" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 37361 } } peers { permanent_uuid: "ed89bfa60b114ee6a11787f6913bf1be" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 37329 } } peers { permanent_uuid: "768749257e7f437693e5f9a234a3db0c" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 38953 } }
I20260809 09:32:33.607671 24418 ts_tablet_manager.cc:1428] T 8d47fee6639b4a57a0f4362cb2083b02 P ed89bfa60b114ee6a11787f6913bf1be: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260809 09:32:33.610986 23623 catalog_manager.cc:4841] TS 768749257e7f437693e5f9a234a3db0c (127.20.125.131:38953): tablet daa39432dbc14ab198ef697006127abd (table test-table [id=856cdbf2690a418f94d612f9abac6c84]) successfully deleted
I20260809 09:32:33.631048 24095 raft_consensus.cc:3037] T 8d47fee6639b4a57a0f4362cb2083b02 P ed89bfa60b114ee6a11787f6913bf1be [term 0 FOLLOWER]: Advancing to term 1
I20260809 09:32:33.631389 23880 raft_consensus.cc:1261] T 8d47fee6639b4a57a0f4362cb2083b02 P ff6decf4b31348898adbb75faaab9faf [term 1 FOLLOWER]: Refusing update from remote peer 768749257e7f437693e5f9a234a3db0c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 09:32:33.631960 24434 consensus_queue.cc:1035] T 8d47fee6639b4a57a0f4362cb2083b02 P 768749257e7f437693e5f9a234a3db0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff6decf4b31348898adbb75faaab9faf" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 37361 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 09:32:33.632488 24095 raft_consensus.cc:1261] T 8d47fee6639b4a57a0f4362cb2083b02 P ed89bfa60b114ee6a11787f6913bf1be [term 1 FOLLOWER]: Refusing update from remote peer 768749257e7f437693e5f9a234a3db0c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 09:32:33.632969 24434 consensus_queue.cc:1035] T 8d47fee6639b4a57a0f4362cb2083b02 P 768749257e7f437693e5f9a234a3db0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ed89bfa60b114ee6a11787f6913bf1be" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 37329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260809 09:32:33.651628 24463 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260809 09:32:35.224952 23644 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39006:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 09:32:35.225178 23644 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39006:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 09:32:35.228149 23644 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 22534bf98c554d6c8acdd00f8d2a9a29: Sending DeleteTablet for 3 replicas of tablet 8d47fee6639b4a57a0f4362cb2083b02
I20260809 09:32:35.228907 23851 tablet_service.cc:1514] Processing DeleteTablet for tablet 8d47fee6639b4a57a0f4362cb2083b02 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 09:32:35 UTC) from {username='kudu', principal='kudu/127.20.125.190@KRBTEST.COM'} at 127.0.0.1:35230
I20260809 09:32:35.228916 24288 tablet_service.cc:1514] Processing DeleteTablet for tablet 8d47fee6639b4a57a0f4362cb2083b02 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 09:32:35 UTC) from {username='kudu', principal='kudu/127.20.125.190@KRBTEST.COM'} at 127.0.0.1:45356
I20260809 09:32:35.228957 24067 tablet_service.cc:1514] Processing DeleteTablet for tablet 8d47fee6639b4a57a0f4362cb2083b02 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 09:32:35 UTC) from {username='kudu', principal='kudu/127.20.125.190@KRBTEST.COM'} at 127.0.0.1:48072
I20260809 09:32:35.229324 24544 tablet_replica.cc:330] T 8d47fee6639b4a57a0f4362cb2083b02 P ed89bfa60b114ee6a11787f6913bf1be: stopping tablet replica
I20260809 09:32:35.229547 24544 raft_consensus.cc:2229] T 8d47fee6639b4a57a0f4362cb2083b02 P ed89bfa60b114ee6a11787f6913bf1be [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 09:32:35.229543 24543 tablet_replica.cc:330] T 8d47fee6639b4a57a0f4362cb2083b02 P ff6decf4b31348898adbb75faaab9faf: stopping tablet replica
I20260809 09:32:35.229763 24544 raft_consensus.cc:2258] T 8d47fee6639b4a57a0f4362cb2083b02 P ed89bfa60b114ee6a11787f6913bf1be [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 09:32:35.229744 24543 raft_consensus.cc:2229] T 8d47fee6639b4a57a0f4362cb2083b02 P ff6decf4b31348898adbb75faaab9faf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 09:32:35.229897 24543 raft_consensus.cc:2258] T 8d47fee6639b4a57a0f4362cb2083b02 P ff6decf4b31348898adbb75faaab9faf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 09:32:35.230281 24542 tablet_replica.cc:330] T 8d47fee6639b4a57a0f4362cb2083b02 P 768749257e7f437693e5f9a234a3db0c: stopping tablet replica
I20260809 09:32:35.230525 24542 raft_consensus.cc:2229] T 8d47fee6639b4a57a0f4362cb2083b02 P 768749257e7f437693e5f9a234a3db0c [term 1 LEADER]: Raft consensus shutting down.
I20260809 09:32:35.230827 24542 raft_consensus.cc:2258] T 8d47fee6639b4a57a0f4362cb2083b02 P 768749257e7f437693e5f9a234a3db0c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 09:32:35.230854 24544 ts_tablet_manager.cc:1905] T 8d47fee6639b4a57a0f4362cb2083b02 P ed89bfa60b114ee6a11787f6913bf1be: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 09:32:35.231225 24543 ts_tablet_manager.cc:1905] T 8d47fee6639b4a57a0f4362cb2083b02 P ff6decf4b31348898adbb75faaab9faf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 09:32:35.231614 24542 ts_tablet_manager.cc:1905] T 8d47fee6639b4a57a0f4362cb2083b02 P 768749257e7f437693e5f9a234a3db0c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 09:32:35.234558 24544 ts_tablet_manager.cc:1918] T 8d47fee6639b4a57a0f4362cb2083b02 P ed89bfa60b114ee6a11787f6913bf1be: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 09:32:35.234759 24544 log.cc:1196] T 8d47fee6639b4a57a0f4362cb2083b02 P ed89bfa60b114ee6a11787f6913bf1be: Deleting WAL directory at /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786267941944493-20982-0/minicluster-data/ts-1/wal/wals/8d47fee6639b4a57a0f4362cb2083b02
I20260809 09:32:35.235258 24543 ts_tablet_manager.cc:1918] T 8d47fee6639b4a57a0f4362cb2083b02 P ff6decf4b31348898adbb75faaab9faf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 09:32:35.235337 24542 ts_tablet_manager.cc:1918] T 8d47fee6639b4a57a0f4362cb2083b02 P 768749257e7f437693e5f9a234a3db0c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 09:32:35.235402 24544 ts_tablet_manager.cc:1939] T 8d47fee6639b4a57a0f4362cb2083b02 P ed89bfa60b114ee6a11787f6913bf1be: Deleting consensus metadata
I20260809 09:32:35.235436 24542 log.cc:1196] T 8d47fee6639b4a57a0f4362cb2083b02 P 768749257e7f437693e5f9a234a3db0c: Deleting WAL directory at /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786267941944493-20982-0/minicluster-data/ts-2/wal/wals/8d47fee6639b4a57a0f4362cb2083b02
I20260809 09:32:35.235390 24543 log.cc:1196] T 8d47fee6639b4a57a0f4362cb2083b02 P ff6decf4b31348898adbb75faaab9faf: Deleting WAL directory at /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786267941944493-20982-0/minicluster-data/ts-0/wal/wals/8d47fee6639b4a57a0f4362cb2083b02
I20260809 09:32:35.235839 24543 ts_tablet_manager.cc:1939] T 8d47fee6639b4a57a0f4362cb2083b02 P ff6decf4b31348898adbb75faaab9faf: Deleting consensus metadata
I20260809 09:32:35.236359 23626 catalog_manager.cc:4841] TS ed89bfa60b114ee6a11787f6913bf1be (127.20.125.130:37329): tablet 8d47fee6639b4a57a0f4362cb2083b02 (table test-table [id=0d1150d44c15434996c8d26a74c66e28]) successfully deleted
I20260809 09:32:35.236586 23626 catalog_manager.cc:4841] TS ff6decf4b31348898adbb75faaab9faf (127.20.125.129:37361): tablet 8d47fee6639b4a57a0f4362cb2083b02 (table test-table [id=0d1150d44c15434996c8d26a74c66e28]) successfully deleted
I20260809 09:32:35.236956 24542 ts_tablet_manager.cc:1939] T 8d47fee6639b4a57a0f4362cb2083b02 P 768749257e7f437693e5f9a234a3db0c: Deleting consensus metadata
I20260809 09:32:35.237680 23623 catalog_manager.cc:4841] TS 768749257e7f437693e5f9a234a3db0c (127.20.125.131:38953): tablet 8d47fee6639b4a57a0f4362cb2083b02 (table test-table [id=0d1150d44c15434996c8d26a74c66e28]) successfully deleted
Aug 09 09:32:35 dist-test-slave-7kzw krb5kdc[23584](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786267955, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 09:32:35 dist-test-slave-7kzw krb5kdc[23584](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786267955, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.20.125.190@KRBTEST.COM
I20260809 09:32:35.267700 24554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:35.253264 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60212 (local address 127.20.125.190:41195)
0809 09:32:35.253601 (+ 337us) server_negotiation.cc:206] Beginning negotiation
0809 09:32:35.253610 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 09:32:35.253631 (+ 21us) server_negotiation.cc:407] Connection header received
0809 09:32:35.253718 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:32:35.253721 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:32:35.253765 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:32:35.253857 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:32:35.254979 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:35.255784 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:35.256808 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:35.257043 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:35.257134 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:35.263601 (+ 6467us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:32:35.263604 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:32:35.263606 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:32:35.263672 (+ 66us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:32:35.265447 (+ 1775us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:32:35.265959 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:32:35.265962 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:32:35.265964 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:32:35.266070 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:32:35.266364 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:32:35.266367 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:32:35.266368 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:32:35.266819 (+ 451us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:32:35.266977 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0809 09:32:35.267359 (+ 382us) server_negotiation.cc:299] Negotiation successful
0809 09:32:35.267496 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":262,"thread_start_us":167,"threads_started":1}
I20260809 09:32:35.271001 23644 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60212:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 09:32:35.271544 23644 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 09:32:35.277245 23851 tablet_service.cc:1467] Processing CreateTablet for tablet 9e280586b0b44080adf7bf5eda797e4e (DEFAULT_TABLE table=test-table [id=14d0c6c1cb984f04a9fdd14191d4df9f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 09:32:35.277316 24067 tablet_service.cc:1467] Processing CreateTablet for tablet 9e280586b0b44080adf7bf5eda797e4e (DEFAULT_TABLE table=test-table [id=14d0c6c1cb984f04a9fdd14191d4df9f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 09:32:35.277520 24288 tablet_service.cc:1467] Processing CreateTablet for tablet 9e280586b0b44080adf7bf5eda797e4e (DEFAULT_TABLE table=test-table [id=14d0c6c1cb984f04a9fdd14191d4df9f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 09:32:35.277663 24067 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9e280586b0b44080adf7bf5eda797e4e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 09:32:35.277716 23851 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9e280586b0b44080adf7bf5eda797e4e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 09:32:35.277994 24288 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9e280586b0b44080adf7bf5eda797e4e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 09:32:35.280457 24556 tablet_bootstrap.cc:492] T 9e280586b0b44080adf7bf5eda797e4e P 768749257e7f437693e5f9a234a3db0c: Bootstrap starting.
I20260809 09:32:35.280937 24557 tablet_bootstrap.cc:492] T 9e280586b0b44080adf7bf5eda797e4e P ff6decf4b31348898adbb75faaab9faf: Bootstrap starting.
I20260809 09:32:35.281128 24559 tablet_bootstrap.cc:492] T 9e280586b0b44080adf7bf5eda797e4e P ed89bfa60b114ee6a11787f6913bf1be: Bootstrap starting.
I20260809 09:32:35.281849 24557 tablet_bootstrap.cc:654] T 9e280586b0b44080adf7bf5eda797e4e P ff6decf4b31348898adbb75faaab9faf: Neither blocks nor log segments found. Creating new log.
I20260809 09:32:35.281842 24556 tablet_bootstrap.cc:654] T 9e280586b0b44080adf7bf5eda797e4e P 768749257e7f437693e5f9a234a3db0c: Neither blocks nor log segments found. Creating new log.
I20260809 09:32:35.282124 24559 tablet_bootstrap.cc:654] T 9e280586b0b44080adf7bf5eda797e4e P ed89bfa60b114ee6a11787f6913bf1be: Neither blocks nor log segments found. Creating new log.
I20260809 09:32:35.283334 24557 tablet_bootstrap.cc:492] T 9e280586b0b44080adf7bf5eda797e4e P ff6decf4b31348898adbb75faaab9faf: No bootstrap required, opened a new log
I20260809 09:32:35.283380 24556 tablet_bootstrap.cc:492] T 9e280586b0b44080adf7bf5eda797e4e P 768749257e7f437693e5f9a234a3db0c: No bootstrap required, opened a new log
I20260809 09:32:35.283440 24557 ts_tablet_manager.cc:1397] T 9e280586b0b44080adf7bf5eda797e4e P ff6decf4b31348898adbb75faaab9faf: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 09:32:35.283478 24556 ts_tablet_manager.cc:1397] T 9e280586b0b44080adf7bf5eda797e4e P 768749257e7f437693e5f9a234a3db0c: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20260809 09:32:35.283488 24559 tablet_bootstrap.cc:492] T 9e280586b0b44080adf7bf5eda797e4e P ed89bfa60b114ee6a11787f6913bf1be: No bootstrap required, opened a new log
I20260809 09:32:35.283571 24559 ts_tablet_manager.cc:1397] T 9e280586b0b44080adf7bf5eda797e4e P ed89bfa60b114ee6a11787f6913bf1be: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 09:32:35.283989 24556 raft_consensus.cc:348] T 9e280586b0b44080adf7bf5eda797e4e P 768749257e7f437693e5f9a234a3db0c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff6decf4b31348898adbb75faaab9faf" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 37361 } } peers { permanent_uuid: "ed89bfa60b114ee6a11787f6913bf1be" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 37329 } } peers { permanent_uuid: "768749257e7f437693e5f9a234a3db0c" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 38953 } }
I20260809 09:32:35.283994 24557 raft_consensus.cc:348] T 9e280586b0b44080adf7bf5eda797e4e P ff6decf4b31348898adbb75faaab9faf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff6decf4b31348898adbb75faaab9faf" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 37361 } } peers { permanent_uuid: "ed89bfa60b114ee6a11787f6913bf1be" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 37329 } } peers { permanent_uuid: "768749257e7f437693e5f9a234a3db0c" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 38953 } }
I20260809 09:32:35.284090 24556 raft_consensus.cc:374] T 9e280586b0b44080adf7bf5eda797e4e P 768749257e7f437693e5f9a234a3db0c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 09:32:35.284114 24556 raft_consensus.cc:729] T 9e280586b0b44080adf7bf5eda797e4e P 768749257e7f437693e5f9a234a3db0c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 768749257e7f437693e5f9a234a3db0c, State: Initialized, Role: FOLLOWER
I20260809 09:32:35.284116 24557 raft_consensus.cc:374] T 9e280586b0b44080adf7bf5eda797e4e P ff6decf4b31348898adbb75faaab9faf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 09:32:35.284154 24557 raft_consensus.cc:729] T 9e280586b0b44080adf7bf5eda797e4e P ff6decf4b31348898adbb75faaab9faf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff6decf4b31348898adbb75faaab9faf, State: Initialized, Role: FOLLOWER
I20260809 09:32:35.284266 24556 consensus_queue.cc:260] T 9e280586b0b44080adf7bf5eda797e4e P 768749257e7f437693e5f9a234a3db0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff6decf4b31348898adbb75faaab9faf" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 37361 } } peers { permanent_uuid: "ed89bfa60b114ee6a11787f6913bf1be" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 37329 } } peers { permanent_uuid: "768749257e7f437693e5f9a234a3db0c" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 38953 } }
I20260809 09:32:35.284300 24557 consensus_queue.cc:260] T 9e280586b0b44080adf7bf5eda797e4e P ff6decf4b31348898adbb75faaab9faf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff6decf4b31348898adbb75faaab9faf" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 37361 } } peers { permanent_uuid: "ed89bfa60b114ee6a11787f6913bf1be" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 37329 } } peers { permanent_uuid: "768749257e7f437693e5f9a234a3db0c" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 38953 } }
I20260809 09:32:35.284358 24559 raft_consensus.cc:348] T 9e280586b0b44080adf7bf5eda797e4e P ed89bfa60b114ee6a11787f6913bf1be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff6decf4b31348898adbb75faaab9faf" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 37361 } } peers { permanent_uuid: "ed89bfa60b114ee6a11787f6913bf1be" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 37329 } } peers { permanent_uuid: "768749257e7f437693e5f9a234a3db0c" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 38953 } }
I20260809 09:32:35.284469 24559 raft_consensus.cc:374] T 9e280586b0b44080adf7bf5eda797e4e P ed89bfa60b114ee6a11787f6913bf1be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 09:32:35.284504 24559 raft_consensus.cc:729] T 9e280586b0b44080adf7bf5eda797e4e P ed89bfa60b114ee6a11787f6913bf1be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed89bfa60b114ee6a11787f6913bf1be, State: Initialized, Role: FOLLOWER
I20260809 09:32:35.284591 24556 ts_tablet_manager.cc:1428] T 9e280586b0b44080adf7bf5eda797e4e P 768749257e7f437693e5f9a234a3db0c: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260809 09:32:35.284641 24557 ts_tablet_manager.cc:1428] T 9e280586b0b44080adf7bf5eda797e4e P ff6decf4b31348898adbb75faaab9faf: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260809 09:32:35.284667 24559 consensus_queue.cc:260] T 9e280586b0b44080adf7bf5eda797e4e P ed89bfa60b114ee6a11787f6913bf1be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff6decf4b31348898adbb75faaab9faf" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 37361 } } peers { permanent_uuid: "ed89bfa60b114ee6a11787f6913bf1be" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 37329 } } peers { permanent_uuid: "768749257e7f437693e5f9a234a3db0c" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 38953 } }
I20260809 09:32:35.285116 24559 ts_tablet_manager.cc:1428] T 9e280586b0b44080adf7bf5eda797e4e P ed89bfa60b114ee6a11787f6913bf1be: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 09:32:35.434054 24531 raft_consensus.cc:482] T 9e280586b0b44080adf7bf5eda797e4e P 768749257e7f437693e5f9a234a3db0c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 09:32:35.434221 24531 raft_consensus.cc:504] T 9e280586b0b44080adf7bf5eda797e4e P 768749257e7f437693e5f9a234a3db0c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff6decf4b31348898adbb75faaab9faf" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 37361 } } peers { permanent_uuid: "ed89bfa60b114ee6a11787f6913bf1be" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 37329 } } peers { permanent_uuid: "768749257e7f437693e5f9a234a3db0c" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 38953 } }
I20260809 09:32:35.434768 24531 leader_election.cc:288] T 9e280586b0b44080adf7bf5eda797e4e P 768749257e7f437693e5f9a234a3db0c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ff6decf4b31348898adbb75faaab9faf (127.20.125.129:37361), ed89bfa60b114ee6a11787f6913bf1be (127.20.125.130:37329)
I20260809 09:32:35.435304 23880 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e280586b0b44080adf7bf5eda797e4e" candidate_uuid: "768749257e7f437693e5f9a234a3db0c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff6decf4b31348898adbb75faaab9faf" is_pre_election: true
I20260809 09:32:35.435472 24098 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e280586b0b44080adf7bf5eda797e4e" candidate_uuid: "768749257e7f437693e5f9a234a3db0c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed89bfa60b114ee6a11787f6913bf1be" is_pre_election: true
I20260809 09:32:35.435526 23880 raft_consensus.cc:2454] T 9e280586b0b44080adf7bf5eda797e4e P ff6decf4b31348898adbb75faaab9faf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 768749257e7f437693e5f9a234a3db0c in term 0.
I20260809 09:32:35.435631 24098 raft_consensus.cc:2454] T 9e280586b0b44080adf7bf5eda797e4e P ed89bfa60b114ee6a11787f6913bf1be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 768749257e7f437693e5f9a234a3db0c in term 0.
I20260809 09:32:35.435940 24211 leader_election.cc:302] T 9e280586b0b44080adf7bf5eda797e4e P 768749257e7f437693e5f9a234a3db0c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 768749257e7f437693e5f9a234a3db0c, ff6decf4b31348898adbb75faaab9faf; no voters:
I20260809 09:32:35.436312 24531 raft_consensus.cc:2781] T 9e280586b0b44080adf7bf5eda797e4e P 768749257e7f437693e5f9a234a3db0c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 09:32:35.436395 24531 raft_consensus.cc:482] T 9e280586b0b44080adf7bf5eda797e4e P 768749257e7f437693e5f9a234a3db0c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 09:32:35.436441 24531 raft_consensus.cc:3037] T 9e280586b0b44080adf7bf5eda797e4e P 768749257e7f437693e5f9a234a3db0c [term 0 FOLLOWER]: Advancing to term 1
I20260809 09:32:35.437803 24531 raft_consensus.cc:504] T 9e280586b0b44080adf7bf5eda797e4e P 768749257e7f437693e5f9a234a3db0c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff6decf4b31348898adbb75faaab9faf" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 37361 } } peers { permanent_uuid: "ed89bfa60b114ee6a11787f6913bf1be" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 37329 } } peers { permanent_uuid: "768749257e7f437693e5f9a234a3db0c" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 38953 } }
I20260809 09:32:35.438195 24531 leader_election.cc:288] T 9e280586b0b44080adf7bf5eda797e4e P 768749257e7f437693e5f9a234a3db0c [CANDIDATE]: Term 1 election: Requested vote from peers ff6decf4b31348898adbb75faaab9faf (127.20.125.129:37361), ed89bfa60b114ee6a11787f6913bf1be (127.20.125.130:37329)
I20260809 09:32:35.438850 24098 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e280586b0b44080adf7bf5eda797e4e" candidate_uuid: "768749257e7f437693e5f9a234a3db0c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed89bfa60b114ee6a11787f6913bf1be"
I20260809 09:32:35.438856 23880 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e280586b0b44080adf7bf5eda797e4e" candidate_uuid: "768749257e7f437693e5f9a234a3db0c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff6decf4b31348898adbb75faaab9faf"
I20260809 09:32:35.439014 24098 raft_consensus.cc:3037] T 9e280586b0b44080adf7bf5eda797e4e P ed89bfa60b114ee6a11787f6913bf1be [term 0 FOLLOWER]: Advancing to term 1
I20260809 09:32:35.439013 23880 raft_consensus.cc:3037] T 9e280586b0b44080adf7bf5eda797e4e P ff6decf4b31348898adbb75faaab9faf [term 0 FOLLOWER]: Advancing to term 1
I20260809 09:32:35.440300 24098 raft_consensus.cc:2454] T 9e280586b0b44080adf7bf5eda797e4e P ed89bfa60b114ee6a11787f6913bf1be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 768749257e7f437693e5f9a234a3db0c in term 1.
I20260809 09:32:35.440869 24211 leader_election.cc:302] T 9e280586b0b44080adf7bf5eda797e4e P 768749257e7f437693e5f9a234a3db0c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 768749257e7f437693e5f9a234a3db0c, ed89bfa60b114ee6a11787f6913bf1be; no voters:
I20260809 09:32:35.441105 24531 raft_consensus.cc:2781] T 9e280586b0b44080adf7bf5eda797e4e P 768749257e7f437693e5f9a234a3db0c [term 1 FOLLOWER]: Leader election won for term 1
I20260809 09:32:35.441269 24531 raft_consensus.cc:686] T 9e280586b0b44080adf7bf5eda797e4e P 768749257e7f437693e5f9a234a3db0c [term 1 LEADER]: Becoming Leader. State: Replica: 768749257e7f437693e5f9a234a3db0c, State: Running, Role: LEADER
I20260809 09:32:35.441447 24531 consensus_queue.cc:237] T 9e280586b0b44080adf7bf5eda797e4e P 768749257e7f437693e5f9a234a3db0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff6decf4b31348898adbb75faaab9faf" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 37361 } } peers { permanent_uuid: "ed89bfa60b114ee6a11787f6913bf1be" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 37329 } } peers { permanent_uuid: "768749257e7f437693e5f9a234a3db0c" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 38953 } }
I20260809 09:32:35.442749 23880 raft_consensus.cc:2454] T 9e280586b0b44080adf7bf5eda797e4e P ff6decf4b31348898adbb75faaab9faf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 768749257e7f437693e5f9a234a3db0c in term 1.
I20260809 09:32:35.443679 23642 catalog_manager.cc:5495] T 9e280586b0b44080adf7bf5eda797e4e P 768749257e7f437693e5f9a234a3db0c reported cstate change: term changed from 0 to 1, leader changed from <none> to 768749257e7f437693e5f9a234a3db0c (127.20.125.131). New cstate: current_term: 1 leader_uuid: "768749257e7f437693e5f9a234a3db0c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff6decf4b31348898adbb75faaab9faf" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 37361 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ed89bfa60b114ee6a11787f6913bf1be" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 37329 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "768749257e7f437693e5f9a234a3db0c" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 38953 } health_report { overall_health: HEALTHY } } }
I20260809 09:32:35.489207 24573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:35.485547 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57362 (local address 127.20.125.131:38953)
0809 09:32:35.485912 (+ 365us) server_negotiation.cc:206] Beginning negotiation
0809 09:32:35.485924 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 09:32:35.485939 (+ 15us) server_negotiation.cc:407] Connection header received
0809 09:32:35.485985 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:32:35.485988 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:32:35.486039 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:32:35.486129 (+ 90us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 09:32:35.486448 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:35.487146 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:35.488038 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:35.488291 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:35.488415 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:35.488709 (+ 294us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 09:32:35.488814 (+ 105us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 09:32:35.488898 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0809 09:32:35.488992 (+ 94us) server_negotiation.cc:299] Negotiation successful
0809 09:32:35.489046 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":265,"thread_start_us":104,"threads_started":1}
I20260809 09:32:35.496126 24098 raft_consensus.cc:1261] T 9e280586b0b44080adf7bf5eda797e4e P ed89bfa60b114ee6a11787f6913bf1be [term 1 FOLLOWER]: Refusing update from remote peer 768749257e7f437693e5f9a234a3db0c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 09:32:35.496455 23880 raft_consensus.cc:1261] T 9e280586b0b44080adf7bf5eda797e4e P ff6decf4b31348898adbb75faaab9faf [term 1 FOLLOWER]: Refusing update from remote peer 768749257e7f437693e5f9a234a3db0c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 09:32:35.496579 24531 consensus_queue.cc:1035] T 9e280586b0b44080adf7bf5eda797e4e P 768749257e7f437693e5f9a234a3db0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ed89bfa60b114ee6a11787f6913bf1be" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 37329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 09:32:35.496897 24535 consensus_queue.cc:1035] T 9e280586b0b44080adf7bf5eda797e4e P 768749257e7f437693e5f9a234a3db0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff6decf4b31348898adbb75faaab9faf" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 37361 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 09:32:35.530298 23643 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60212:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 09:32:35.530454 23643 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60212:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 09:32:35.532819 23643 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 22534bf98c554d6c8acdd00f8d2a9a29: Sending DeleteTablet for 3 replicas of tablet 9e280586b0b44080adf7bf5eda797e4e
I20260809 09:32:35.533639 24067 tablet_service.cc:1514] Processing DeleteTablet for tablet 9e280586b0b44080adf7bf5eda797e4e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 09:32:35 UTC) from {username='kudu', principal='kudu/127.20.125.190@KRBTEST.COM'} at 127.0.0.1:48072
I20260809 09:32:35.533890 24544 tablet_replica.cc:330] T 9e280586b0b44080adf7bf5eda797e4e P ed89bfa60b114ee6a11787f6913bf1be: stopping tablet replica
I20260809 09:32:35.533950 23851 tablet_service.cc:1514] Processing DeleteTablet for tablet 9e280586b0b44080adf7bf5eda797e4e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 09:32:35 UTC) from {username='kudu', principal='kudu/127.20.125.190@KRBTEST.COM'} at 127.0.0.1:35230
I20260809 09:32:35.534047 24544 raft_consensus.cc:2229] T 9e280586b0b44080adf7bf5eda797e4e P ed89bfa60b114ee6a11787f6913bf1be [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 09:32:35.534138 24543 tablet_replica.cc:330] T 9e280586b0b44080adf7bf5eda797e4e P ff6decf4b31348898adbb75faaab9faf: stopping tablet replica
I20260809 09:32:35.534197 24544 raft_consensus.cc:2258] T 9e280586b0b44080adf7bf5eda797e4e P ed89bfa60b114ee6a11787f6913bf1be [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 09:32:35.534317 24543 raft_consensus.cc:2229] T 9e280586b0b44080adf7bf5eda797e4e P ff6decf4b31348898adbb75faaab9faf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 09:32:35.534504 24543 raft_consensus.cc:2258] T 9e280586b0b44080adf7bf5eda797e4e P ff6decf4b31348898adbb75faaab9faf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 09:32:35.534433 24288 tablet_service.cc:1514] Processing DeleteTablet for tablet 9e280586b0b44080adf7bf5eda797e4e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 09:32:35 UTC) from {username='kudu', principal='kudu/127.20.125.190@KRBTEST.COM'} at 127.0.0.1:45356
I20260809 09:32:35.534595 23643 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60212:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 09:32:35.534768 24542 tablet_replica.cc:330] T 9e280586b0b44080adf7bf5eda797e4e P 768749257e7f437693e5f9a234a3db0c: stopping tablet replica
I20260809 09:32:35.534912 24542 raft_consensus.cc:2229] T 9e280586b0b44080adf7bf5eda797e4e P 768749257e7f437693e5f9a234a3db0c [term 1 LEADER]: Raft consensus shutting down.
W20260809 09:32:35.535076 23643 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 09:32:35.535120 24542 raft_consensus.cc:2258] T 9e280586b0b44080adf7bf5eda797e4e P 768749257e7f437693e5f9a234a3db0c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 09:32:35.535406 24544 ts_tablet_manager.cc:1905] T 9e280586b0b44080adf7bf5eda797e4e P ed89bfa60b114ee6a11787f6913bf1be: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 09:32:35.538050 24544 ts_tablet_manager.cc:1918] T 9e280586b0b44080adf7bf5eda797e4e P ed89bfa60b114ee6a11787f6913bf1be: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 09:32:35.538122 24544 log.cc:1196] T 9e280586b0b44080adf7bf5eda797e4e P ed89bfa60b114ee6a11787f6913bf1be: Deleting WAL directory at /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786267941944493-20982-0/minicluster-data/ts-1/wal/wals/9e280586b0b44080adf7bf5eda797e4e
I20260809 09:32:35.538398 24544 ts_tablet_manager.cc:1939] T 9e280586b0b44080adf7bf5eda797e4e P ed89bfa60b114ee6a11787f6913bf1be: Deleting consensus metadata
I20260809 09:32:35.539338 23626 catalog_manager.cc:4841] TS ed89bfa60b114ee6a11787f6913bf1be (127.20.125.130:37329): tablet 9e280586b0b44080adf7bf5eda797e4e (table test-table [id=14d0c6c1cb984f04a9fdd14191d4df9f]) successfully deleted
I20260809 09:32:35.539757 24542 ts_tablet_manager.cc:1905] T 9e280586b0b44080adf7bf5eda797e4e P 768749257e7f437693e5f9a234a3db0c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 09:32:35.540112 24543 ts_tablet_manager.cc:1905] T 9e280586b0b44080adf7bf5eda797e4e P ff6decf4b31348898adbb75faaab9faf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 09:32:35.542443 24542 ts_tablet_manager.cc:1918] T 9e280586b0b44080adf7bf5eda797e4e P 768749257e7f437693e5f9a234a3db0c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 09:32:35.542577 24542 log.cc:1196] T 9e280586b0b44080adf7bf5eda797e4e P 768749257e7f437693e5f9a234a3db0c: Deleting WAL directory at /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786267941944493-20982-0/minicluster-data/ts-2/wal/wals/9e280586b0b44080adf7bf5eda797e4e
I20260809 09:32:35.542814 24543 ts_tablet_manager.cc:1918] T 9e280586b0b44080adf7bf5eda797e4e P ff6decf4b31348898adbb75faaab9faf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 09:32:35.542867 24542 ts_tablet_manager.cc:1939] T 9e280586b0b44080adf7bf5eda797e4e P 768749257e7f437693e5f9a234a3db0c: Deleting consensus metadata
I20260809 09:32:35.542968 24543 log.cc:1196] T 9e280586b0b44080adf7bf5eda797e4e P ff6decf4b31348898adbb75faaab9faf: Deleting WAL directory at /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786267941944493-20982-0/minicluster-data/ts-0/wal/wals/9e280586b0b44080adf7bf5eda797e4e
I20260809 09:32:35.543256 24543 ts_tablet_manager.cc:1939] T 9e280586b0b44080adf7bf5eda797e4e P ff6decf4b31348898adbb75faaab9faf: Deleting consensus metadata
I20260809 09:32:35.543620 23623 catalog_manager.cc:4841] TS 768749257e7f437693e5f9a234a3db0c (127.20.125.131:38953): tablet 9e280586b0b44080adf7bf5eda797e4e (table test-table [id=14d0c6c1cb984f04a9fdd14191d4df9f]) successfully deleted
I20260809 09:32:35.543990 24288 tablet_service.cc:1467] Processing CreateTablet for tablet 6cd5a2346d5f46958d38d10b30b929f6 (DEFAULT_TABLE table=test-table [id=66715c5ebb4346709bd46f61c08533f9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 09:32:35.544050 23626 catalog_manager.cc:4841] TS ff6decf4b31348898adbb75faaab9faf (127.20.125.129:37361): tablet 9e280586b0b44080adf7bf5eda797e4e (table test-table [id=14d0c6c1cb984f04a9fdd14191d4df9f]) successfully deleted
I20260809 09:32:35.544278 24288 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6cd5a2346d5f46958d38d10b30b929f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 09:32:35.544304 24067 tablet_service.cc:1467] Processing CreateTablet for tablet 6cd5a2346d5f46958d38d10b30b929f6 (DEFAULT_TABLE table=test-table [id=66715c5ebb4346709bd46f61c08533f9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 09:32:35.544598 24067 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6cd5a2346d5f46958d38d10b30b929f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 09:32:35.546061 24556 tablet_bootstrap.cc:492] T 6cd5a2346d5f46958d38d10b30b929f6 P 768749257e7f437693e5f9a234a3db0c: Bootstrap starting.
I20260809 09:32:35.546717 24559 tablet_bootstrap.cc:492] T 6cd5a2346d5f46958d38d10b30b929f6 P ed89bfa60b114ee6a11787f6913bf1be: Bootstrap starting.
I20260809 09:32:35.546844 23851 tablet_service.cc:1467] Processing CreateTablet for tablet 6cd5a2346d5f46958d38d10b30b929f6 (DEFAULT_TABLE table=test-table [id=66715c5ebb4346709bd46f61c08533f9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 09:32:35.547034 24556 tablet_bootstrap.cc:654] T 6cd5a2346d5f46958d38d10b30b929f6 P 768749257e7f437693e5f9a234a3db0c: Neither blocks nor log segments found. Creating new log.
I20260809 09:32:35.547135 23851 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6cd5a2346d5f46958d38d10b30b929f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 09:32:35.547714 24559 tablet_bootstrap.cc:654] T 6cd5a2346d5f46958d38d10b30b929f6 P ed89bfa60b114ee6a11787f6913bf1be: Neither blocks nor log segments found. Creating new log.
I20260809 09:32:35.548741 24556 tablet_bootstrap.cc:492] T 6cd5a2346d5f46958d38d10b30b929f6 P 768749257e7f437693e5f9a234a3db0c: No bootstrap required, opened a new log
I20260809 09:32:35.548846 24556 ts_tablet_manager.cc:1397] T 6cd5a2346d5f46958d38d10b30b929f6 P 768749257e7f437693e5f9a234a3db0c: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 09:32:35.548861 24559 tablet_bootstrap.cc:492] T 6cd5a2346d5f46958d38d10b30b929f6 P ed89bfa60b114ee6a11787f6913bf1be: No bootstrap required, opened a new log
I20260809 09:32:35.548945 24559 ts_tablet_manager.cc:1397] T 6cd5a2346d5f46958d38d10b30b929f6 P ed89bfa60b114ee6a11787f6913bf1be: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 09:32:35.549271 24556 raft_consensus.cc:348] T 6cd5a2346d5f46958d38d10b30b929f6 P 768749257e7f437693e5f9a234a3db0c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff6decf4b31348898adbb75faaab9faf" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 37361 } } peers { permanent_uuid: "ed89bfa60b114ee6a11787f6913bf1be" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 37329 } } peers { permanent_uuid: "768749257e7f437693e5f9a234a3db0c" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 38953 } }
I20260809 09:32:35.549317 24559 raft_consensus.cc:348] T 6cd5a2346d5f46958d38d10b30b929f6 P ed89bfa60b114ee6a11787f6913bf1be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff6decf4b31348898adbb75faaab9faf" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 37361 } } peers { permanent_uuid: "ed89bfa60b114ee6a11787f6913bf1be" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 37329 } } peers { permanent_uuid: "768749257e7f437693e5f9a234a3db0c" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 38953 } }
I20260809 09:32:35.549408 24556 raft_consensus.cc:374] T 6cd5a2346d5f46958d38d10b30b929f6 P 768749257e7f437693e5f9a234a3db0c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 09:32:35.549427 24559 raft_consensus.cc:374] T 6cd5a2346d5f46958d38d10b30b929f6 P ed89bfa60b114ee6a11787f6913bf1be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 09:32:35.549467 24556 raft_consensus.cc:729] T 6cd5a2346d5f46958d38d10b30b929f6 P 768749257e7f437693e5f9a234a3db0c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 768749257e7f437693e5f9a234a3db0c, State: Initialized, Role: FOLLOWER
I20260809 09:32:35.549466 24559 raft_consensus.cc:729] T 6cd5a2346d5f46958d38d10b30b929f6 P ed89bfa60b114ee6a11787f6913bf1be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed89bfa60b114ee6a11787f6913bf1be, State: Initialized, Role: FOLLOWER
I20260809 09:32:35.549599 24556 consensus_queue.cc:260] T 6cd5a2346d5f46958d38d10b30b929f6 P 768749257e7f437693e5f9a234a3db0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff6decf4b31348898adbb75faaab9faf" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 37361 } } peers { permanent_uuid: "ed89bfa60b114ee6a11787f6913bf1be" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 37329 } } peers { permanent_uuid: "768749257e7f437693e5f9a234a3db0c" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 38953 } }
I20260809 09:32:35.549614 24559 consensus_queue.cc:260] T 6cd5a2346d5f46958d38d10b30b929f6 P ed89bfa60b114ee6a11787f6913bf1be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff6decf4b31348898adbb75faaab9faf" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 37361 } } peers { permanent_uuid: "ed89bfa60b114ee6a11787f6913bf1be" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 37329 } } peers { permanent_uuid: "768749257e7f437693e5f9a234a3db0c" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 38953 } }
I20260809 09:32:35.549742 24557 tablet_bootstrap.cc:492] T 6cd5a2346d5f46958d38d10b30b929f6 P ff6decf4b31348898adbb75faaab9faf: Bootstrap starting.
I20260809 09:32:35.549907 24559 ts_tablet_manager.cc:1428] T 6cd5a2346d5f46958d38d10b30b929f6 P ed89bfa60b114ee6a11787f6913bf1be: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260809 09:32:35.549911 24556 ts_tablet_manager.cc:1428] T 6cd5a2346d5f46958d38d10b30b929f6 P 768749257e7f437693e5f9a234a3db0c: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260809 09:32:35.550604 24557 tablet_bootstrap.cc:654] T 6cd5a2346d5f46958d38d10b30b929f6 P ff6decf4b31348898adbb75faaab9faf: Neither blocks nor log segments found. Creating new log.
I20260809 09:32:35.551843 24557 tablet_bootstrap.cc:492] T 6cd5a2346d5f46958d38d10b30b929f6 P ff6decf4b31348898adbb75faaab9faf: No bootstrap required, opened a new log
I20260809 09:32:35.551952 24557 ts_tablet_manager.cc:1397] T 6cd5a2346d5f46958d38d10b30b929f6 P ff6decf4b31348898adbb75faaab9faf: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260809 09:32:35.552424 24557 raft_consensus.cc:348] T 6cd5a2346d5f46958d38d10b30b929f6 P ff6decf4b31348898adbb75faaab9faf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff6decf4b31348898adbb75faaab9faf" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 37361 } } peers { permanent_uuid: "ed89bfa60b114ee6a11787f6913bf1be" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 37329 } } peers { permanent_uuid: "768749257e7f437693e5f9a234a3db0c" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 38953 } }
I20260809 09:32:35.552546 24557 raft_consensus.cc:374] T 6cd5a2346d5f46958d38d10b30b929f6 P ff6decf4b31348898adbb75faaab9faf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 09:32:35.552582 24557 raft_consensus.cc:729] T 6cd5a2346d5f46958d38d10b30b929f6 P ff6decf4b31348898adbb75faaab9faf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff6decf4b31348898adbb75faaab9faf, State: Initialized, Role: FOLLOWER
I20260809 09:32:35.552713 24557 consensus_queue.cc:260] T 6cd5a2346d5f46958d38d10b30b929f6 P ff6decf4b31348898adbb75faaab9faf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff6decf4b31348898adbb75faaab9faf" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 37361 } } peers { permanent_uuid: "ed89bfa60b114ee6a11787f6913bf1be" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 37329 } } peers { permanent_uuid: "768749257e7f437693e5f9a234a3db0c" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 38953 } }
I20260809 09:32:35.553028 24557 ts_tablet_manager.cc:1428] T 6cd5a2346d5f46958d38d10b30b929f6 P ff6decf4b31348898adbb75faaab9faf: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260809 09:32:35.800463 24509 raft_consensus.cc:482] T 6cd5a2346d5f46958d38d10b30b929f6 P ff6decf4b31348898adbb75faaab9faf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 09:32:35.800678 24509 raft_consensus.cc:504] T 6cd5a2346d5f46958d38d10b30b929f6 P ff6decf4b31348898adbb75faaab9faf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff6decf4b31348898adbb75faaab9faf" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 37361 } } peers { permanent_uuid: "ed89bfa60b114ee6a11787f6913bf1be" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 37329 } } peers { permanent_uuid: "768749257e7f437693e5f9a234a3db0c" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 38953 } }
I20260809 09:32:35.801244 24509 leader_election.cc:288] T 6cd5a2346d5f46958d38d10b30b929f6 P ff6decf4b31348898adbb75faaab9faf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ed89bfa60b114ee6a11787f6913bf1be (127.20.125.130:37329), 768749257e7f437693e5f9a234a3db0c (127.20.125.131:38953)
I20260809 09:32:35.801986 24317 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6cd5a2346d5f46958d38d10b30b929f6" candidate_uuid: "ff6decf4b31348898adbb75faaab9faf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "768749257e7f437693e5f9a234a3db0c" is_pre_election: true
I20260809 09:32:35.802012 24098 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6cd5a2346d5f46958d38d10b30b929f6" candidate_uuid: "ff6decf4b31348898adbb75faaab9faf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed89bfa60b114ee6a11787f6913bf1be" is_pre_election: true
I20260809 09:32:35.802165 24317 raft_consensus.cc:2454] T 6cd5a2346d5f46958d38d10b30b929f6 P 768749257e7f437693e5f9a234a3db0c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff6decf4b31348898adbb75faaab9faf in term 0.
I20260809 09:32:35.802213 24098 raft_consensus.cc:2454] T 6cd5a2346d5f46958d38d10b30b929f6 P ed89bfa60b114ee6a11787f6913bf1be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff6decf4b31348898adbb75faaab9faf in term 0.
I20260809 09:32:35.802649 23779 leader_election.cc:302] T 6cd5a2346d5f46958d38d10b30b929f6 P ff6decf4b31348898adbb75faaab9faf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 768749257e7f437693e5f9a234a3db0c, ff6decf4b31348898adbb75faaab9faf; no voters:
I20260809 09:32:35.802893 24509 raft_consensus.cc:2781] T 6cd5a2346d5f46958d38d10b30b929f6 P ff6decf4b31348898adbb75faaab9faf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 09:32:35.802961 24509 raft_consensus.cc:482] T 6cd5a2346d5f46958d38d10b30b929f6 P ff6decf4b31348898adbb75faaab9faf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 09:32:35.802989 24509 raft_consensus.cc:3037] T 6cd5a2346d5f46958d38d10b30b929f6 P ff6decf4b31348898adbb75faaab9faf [term 0 FOLLOWER]: Advancing to term 1
I20260809 09:32:35.804097 24509 raft_consensus.cc:504] T 6cd5a2346d5f46958d38d10b30b929f6 P ff6decf4b31348898adbb75faaab9faf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff6decf4b31348898adbb75faaab9faf" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 37361 } } peers { permanent_uuid: "ed89bfa60b114ee6a11787f6913bf1be" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 37329 } } peers { permanent_uuid: "768749257e7f437693e5f9a234a3db0c" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 38953 } }
I20260809 09:32:35.804670 24509 leader_election.cc:288] T 6cd5a2346d5f46958d38d10b30b929f6 P ff6decf4b31348898adbb75faaab9faf [CANDIDATE]: Term 1 election: Requested vote from peers ed89bfa60b114ee6a11787f6913bf1be (127.20.125.130:37329), 768749257e7f437693e5f9a234a3db0c (127.20.125.131:38953)
I20260809 09:32:35.805155 24098 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6cd5a2346d5f46958d38d10b30b929f6" candidate_uuid: "ff6decf4b31348898adbb75faaab9faf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed89bfa60b114ee6a11787f6913bf1be"
I20260809 09:32:35.805167 24317 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6cd5a2346d5f46958d38d10b30b929f6" candidate_uuid: "ff6decf4b31348898adbb75faaab9faf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "768749257e7f437693e5f9a234a3db0c"
I20260809 09:32:35.805328 24317 raft_consensus.cc:3037] T 6cd5a2346d5f46958d38d10b30b929f6 P 768749257e7f437693e5f9a234a3db0c [term 0 FOLLOWER]: Advancing to term 1
I20260809 09:32:35.805331 24098 raft_consensus.cc:3037] T 6cd5a2346d5f46958d38d10b30b929f6 P ed89bfa60b114ee6a11787f6913bf1be [term 0 FOLLOWER]: Advancing to term 1
I20260809 09:32:35.806406 24317 raft_consensus.cc:2454] T 6cd5a2346d5f46958d38d10b30b929f6 P 768749257e7f437693e5f9a234a3db0c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff6decf4b31348898adbb75faaab9faf in term 1.
I20260809 09:32:35.806535 24098 raft_consensus.cc:2454] T 6cd5a2346d5f46958d38d10b30b929f6 P ed89bfa60b114ee6a11787f6913bf1be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff6decf4b31348898adbb75faaab9faf in term 1.
I20260809 09:32:35.806849 23779 leader_election.cc:302] T 6cd5a2346d5f46958d38d10b30b929f6 P ff6decf4b31348898adbb75faaab9faf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 768749257e7f437693e5f9a234a3db0c, ff6decf4b31348898adbb75faaab9faf; no voters:
I20260809 09:32:35.807114 24509 raft_consensus.cc:2781] T 6cd5a2346d5f46958d38d10b30b929f6 P ff6decf4b31348898adbb75faaab9faf [term 1 FOLLOWER]: Leader election won for term 1
I20260809 09:32:35.807252 24509 raft_consensus.cc:686] T 6cd5a2346d5f46958d38d10b30b929f6 P ff6decf4b31348898adbb75faaab9faf [term 1 LEADER]: Becoming Leader. State: Replica: ff6decf4b31348898adbb75faaab9faf, State: Running, Role: LEADER
I20260809 09:32:35.807446 24509 consensus_queue.cc:237] T 6cd5a2346d5f46958d38d10b30b929f6 P ff6decf4b31348898adbb75faaab9faf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff6decf4b31348898adbb75faaab9faf" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 37361 } } peers { permanent_uuid: "ed89bfa60b114ee6a11787f6913bf1be" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 37329 } } peers { permanent_uuid: "768749257e7f437693e5f9a234a3db0c" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 38953 } }
I20260809 09:32:35.809396 23642 catalog_manager.cc:5495] T 6cd5a2346d5f46958d38d10b30b929f6 P ff6decf4b31348898adbb75faaab9faf reported cstate change: term changed from 0 to 1, leader changed from <none> to ff6decf4b31348898adbb75faaab9faf (127.20.125.129). New cstate: current_term: 1 leader_uuid: "ff6decf4b31348898adbb75faaab9faf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff6decf4b31348898adbb75faaab9faf" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 37361 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ed89bfa60b114ee6a11787f6913bf1be" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 37329 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "768749257e7f437693e5f9a234a3db0c" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 38953 } health_report { overall_health: UNKNOWN } } }
I20260809 09:32:35.855489 24585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:35.851374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48198 (local address 127.20.125.129:37361)
0809 09:32:35.851774 (+ 400us) server_negotiation.cc:206] Beginning negotiation
0809 09:32:35.851783 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 09:32:35.851806 (+ 23us) server_negotiation.cc:407] Connection header received
0809 09:32:35.851851 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:32:35.851853 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:32:35.851899 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:32:35.851985 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 09:32:35.852365 (+ 380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:35.853029 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:35.853977 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:35.854224 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:35.854373 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:35.854788 (+ 415us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 09:32:35.854882 (+ 94us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 09:32:35.854952 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0809 09:32:35.855222 (+ 270us) server_negotiation.cc:299] Negotiation successful
0809 09:32:35.855283 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":289,"thread_start_us":198,"threads_started":1}
I20260809 09:32:35.866247 24585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:35.863158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.129:56413 (local address 127.20.125.129:37361)
0809 09:32:35.863293 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0809 09:32:35.863296 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 09:32:35.863461 (+ 165us) server_negotiation.cc:407] Connection header received
0809 09:32:35.863555 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:32:35.863557 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:32:35.863601 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:32:35.863668 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 09:32:35.863915 (+ 247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:35.864412 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:35.865303 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:35.865508 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:35.865585 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:35.865839 (+ 254us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 09:32:35.865904 (+ 65us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 09:32:35.865958 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0809 09:32:35.866058 (+ 100us) server_negotiation.cc:299] Negotiation successful
0809 09:32:35.866107 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260809 09:32:35.866256 24588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:35.863027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.129:37361 (local address 127.20.125.129:56413)
0809 09:32:35.863369 (+ 342us) negotiation.cc:107] Waiting for socket to connect
0809 09:32:35.863383 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 09:32:35.863484 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:32:35.863708 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:32:35.863711 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:32:35.863721 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 09:32:35.863816 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:32:35.863821 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:32:35.864540 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:32:35.864550 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:32:35.865187 (+ 637us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:32:35.865196 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:32:35.865610 (+ 414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:32:35.865613 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:32:35.865705 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:35.865731 (+ 26us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 09:32:35.865988 (+ 257us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 09:32:35.865993 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 09:32:35.866044 (+ 51us) client_negotiation.cc:240] Negotiation successful
0809 09:32:35.866109 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":250,"thread_start_us":158,"threads_started":1}
I20260809 09:32:35.873606 24098 raft_consensus.cc:1261] T 6cd5a2346d5f46958d38d10b30b929f6 P ed89bfa60b114ee6a11787f6913bf1be [term 1 FOLLOWER]: Refusing update from remote peer ff6decf4b31348898adbb75faaab9faf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 09:32:35.873629 24317 raft_consensus.cc:1261] T 6cd5a2346d5f46958d38d10b30b929f6 P 768749257e7f437693e5f9a234a3db0c [term 1 FOLLOWER]: Refusing update from remote peer ff6decf4b31348898adbb75faaab9faf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 09:32:35.874058 24509 consensus_queue.cc:1035] T 6cd5a2346d5f46958d38d10b30b929f6 P ff6decf4b31348898adbb75faaab9faf [LEADER]: Connected to new peer: Peer: permanent_uuid: "768749257e7f437693e5f9a234a3db0c" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 38953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 09:32:35.874344 24523 consensus_queue.cc:1035] T 6cd5a2346d5f46958d38d10b30b929f6 P ff6decf4b31348898adbb75faaab9faf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ed89bfa60b114ee6a11787f6913bf1be" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 37329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260809 09:32:35.889484 24513 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260809 09:32:37.449823 23642 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60212:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 09:32:37.449977 23642 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60212:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 09:32:37.452531 23642 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 22534bf98c554d6c8acdd00f8d2a9a29: Sending DeleteTablet for 3 replicas of tablet 6cd5a2346d5f46958d38d10b30b929f6
I20260809 09:32:37.453104 23850 tablet_service.cc:1514] Processing DeleteTablet for tablet 6cd5a2346d5f46958d38d10b30b929f6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 09:32:37 UTC) from {username='kudu', principal='kudu/127.20.125.190@KRBTEST.COM'} at 127.0.0.1:35230
I20260809 09:32:37.453183 24067 tablet_service.cc:1514] Processing DeleteTablet for tablet 6cd5a2346d5f46958d38d10b30b929f6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 09:32:37 UTC) from {username='kudu', principal='kudu/127.20.125.190@KRBTEST.COM'} at 127.0.0.1:48072
I20260809 09:32:37.453269 24288 tablet_service.cc:1514] Processing DeleteTablet for tablet 6cd5a2346d5f46958d38d10b30b929f6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 09:32:37 UTC) from {username='kudu', principal='kudu/127.20.125.190@KRBTEST.COM'} at 127.0.0.1:45356
I20260809 09:32:37.453464 24627 tablet_replica.cc:330] T 6cd5a2346d5f46958d38d10b30b929f6 P ff6decf4b31348898adbb75faaab9faf: stopping tablet replica
I20260809 09:32:37.453594 24627 raft_consensus.cc:2229] T 6cd5a2346d5f46958d38d10b30b929f6 P ff6decf4b31348898adbb75faaab9faf [term 1 LEADER]: Raft consensus shutting down.
I20260809 09:32:37.453598 24628 tablet_replica.cc:330] T 6cd5a2346d5f46958d38d10b30b929f6 P ed89bfa60b114ee6a11787f6913bf1be: stopping tablet replica
I20260809 09:32:37.453699 20982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu with pid 23757
I20260809 09:32:37.453732 24628 raft_consensus.cc:2229] T 6cd5a2346d5f46958d38d10b30b929f6 P ed89bfa60b114ee6a11787f6913bf1be [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 09:32:37.456244 24628 raft_consensus.cc:2258] T 6cd5a2346d5f46958d38d10b30b929f6 P ed89bfa60b114ee6a11787f6913bf1be [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 09:32:37.456494 24631 tablet_replica.cc:330] T 6cd5a2346d5f46958d38d10b30b929f6 P 768749257e7f437693e5f9a234a3db0c: stopping tablet replica
I20260809 09:32:37.456626 24631 raft_consensus.cc:2229] T 6cd5a2346d5f46958d38d10b30b929f6 P 768749257e7f437693e5f9a234a3db0c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 09:32:37.456727 24631 raft_consensus.cc:2258] T 6cd5a2346d5f46958d38d10b30b929f6 P 768749257e7f437693e5f9a234a3db0c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 09:32:37.459647 24628 ts_tablet_manager.cc:1905] T 6cd5a2346d5f46958d38d10b30b929f6 P ed89bfa60b114ee6a11787f6913bf1be: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 09:32:37.461922 24631 ts_tablet_manager.cc:1905] T 6cd5a2346d5f46958d38d10b30b929f6 P 768749257e7f437693e5f9a234a3db0c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 09:32:37.463369 24628 ts_tablet_manager.cc:1918] T 6cd5a2346d5f46958d38d10b30b929f6 P ed89bfa60b114ee6a11787f6913bf1be: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 09:32:37.463464 24628 log.cc:1196] T 6cd5a2346d5f46958d38d10b30b929f6 P ed89bfa60b114ee6a11787f6913bf1be: Deleting WAL directory at /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786267941944493-20982-0/minicluster-data/ts-1/wal/wals/6cd5a2346d5f46958d38d10b30b929f6
I20260809 09:32:37.463769 24628 ts_tablet_manager.cc:1939] T 6cd5a2346d5f46958d38d10b30b929f6 P ed89bfa60b114ee6a11787f6913bf1be: Deleting consensus metadata
I20260809 09:32:37.464221 24631 ts_tablet_manager.cc:1918] T 6cd5a2346d5f46958d38d10b30b929f6 P 768749257e7f437693e5f9a234a3db0c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 09:32:37.464318 24631 log.cc:1196] T 6cd5a2346d5f46958d38d10b30b929f6 P 768749257e7f437693e5f9a234a3db0c: Deleting WAL directory at /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786267941944493-20982-0/minicluster-data/ts-2/wal/wals/6cd5a2346d5f46958d38d10b30b929f6
I20260809 09:32:37.464632 24631 ts_tablet_manager.cc:1939] T 6cd5a2346d5f46958d38d10b30b929f6 P 768749257e7f437693e5f9a234a3db0c: Deleting consensus metadata
I20260809 09:32:37.464643 23626 catalog_manager.cc:4841] TS ed89bfa60b114ee6a11787f6913bf1be (127.20.125.130:37329): tablet 6cd5a2346d5f46958d38d10b30b929f6 (table test-table [id=66715c5ebb4346709bd46f61c08533f9]) successfully deleted
I20260809 09:32:37.465298 23623 catalog_manager.cc:4841] TS 768749257e7f437693e5f9a234a3db0c (127.20.125.131:38953): tablet 6cd5a2346d5f46958d38d10b30b929f6 (table test-table [id=66715c5ebb4346709bd46f61c08533f9]) successfully deleted
W20260809 09:32:37.469813 23626 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.20.125.129:37361 (error 108)
I20260809 09:32:37.470116 20982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu with pid 23965
I20260809 09:32:37.475337 24637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:37.474166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.129:37361 (local address 127.0.0.1:48200)
0809 09:32:37.475116 (+ 950us) negotiation.cc:107] Waiting for socket to connect
0809 09:32:37.475207 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.129:37361: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":848,"thread_start_us":646,"threads_started":1}
W20260809 09:32:37.475443 23626 catalog_manager.cc:4584] TS ff6decf4b31348898adbb75faaab9faf (127.20.125.129:37361): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 6cd5a2346d5f46958d38d10b30b929f6: Network error: Client connection negotiation failed: client connection to 127.20.125.129:37361: connect: Connection refused (error 111)
I20260809 09:32:37.484892 20982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu with pid 24180
I20260809 09:32:37.500056 20982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu with pid 23600
2026-08-09T09:32:37Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (6846 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786267941944493-20982-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 09:32:37 dist-test-slave-7kzw krb5kdc[24641](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 09:32:37 dist-test-slave-7kzw krb5kdc[24641](info): set up 2 sockets
Aug 09 09:32:37 dist-test-slave-7kzw krb5kdc[24641](info): commencing operation
krb5kdc: starting...
W20260809 09:32:38.735282 20982 mini_kdc.cc:121] Time spent starting KDC: real 1.206s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786267941944493-20982-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786267941944493-20982-0/krb5kdc/test-user.keytab.
Aug 09 09:32:38 dist-test-slave-7kzw krb5kdc[24641](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786267958, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T09:32:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T09:32:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.125.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.125.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.125.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.190.keytab.
Entry for principal kudu/127.20.125.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.190.keytab.
I20260809 09:32:38.869725 20982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
/tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786267941944493-20982-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786267941944493-20982-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786267941944493-20982-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786267941944493-20982-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.125.190:35947
--webserver_interface=127.20.125.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.190.keytab
--principal=kudu/127.20.125.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.125.148:37103
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.125.190:35947
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786267941944493-20982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786267941944493-20982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786267941944493-20982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 09:32:38.983451 24660 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 09:32:38.983788 24660 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 09:32:38.983870 24660 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 09:32:38.987951 24660 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 09:32:38.988034 24660 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 09:32:38.988080 24660 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 09:32:38.988137 24660 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 09:32:38.988174 24660 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 09:32:38.994132 24660 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.125.148:37103
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786267941944493-20982-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786267941944493-20982-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.125.190:35947
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.190.keytab
--principal=kudu/127.20.125.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.125.190:35947
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786267941944493-20982-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.125.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24660
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786267941944493-20982-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 08:38:55 UTC on ccycloud.quasar-njjleu.root.comops.site
build id 4575935
I20260809 09:32:38.995441 24660 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 09:32:38.996598 24660 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 09:32:39.005939 24673 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:32:39.005986 24669 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:32:39.006028 24660 server_base.cc:1034] running on GCE node
W20260809 09:32:39.005940 24670 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:32:39.006991 24660 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 09:32:39.008158 24660 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 09:32:39.009339 24660 hybrid_clock.cc:648] HybridClock initialized: now 1786267959009326 us; error 27 us; skew 500 ppm
Aug 09 09:32:39 dist-test-slave-7kzw krb5kdc[24641](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786267959, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 09:32:39.012378 24660 init.cc:377] Logged in from keytab as kudu/127.20.125.190@KRBTEST.COM (short username kudu)
I20260809 09:32:39.013296 24660 webserver.cc:460] Webserver started at http://127.20.125.190:46251/ using document root <none> and password file <none>
I20260809 09:32:39.013877 24660 fs_manager.cc:359] Metadata directory not provided
I20260809 09:32:39.013995 24660 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 09:32:39.014209 24660 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 09:32:39.016281 24660 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786267941944493-20982-0/minicluster-data/master-0/data/instance:
uuid: "26e0fa89ce214d20affe757dee8279a2"
format_stamp: "Formatted at 2026-08-09 09:32:39 on dist-test-slave-7kzw"
server_key: "26e5380779e3a53d0c1c4ada0a79208a"
server_key_iv: "03c765abf25be327bcdc1575c6a38ad9"
server_key_version: "encryptionkey@0"
I20260809 09:32:39.016867 24660 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786267941944493-20982-0/minicluster-data/master-0/wal/instance:
uuid: "26e0fa89ce214d20affe757dee8279a2"
format_stamp: "Formatted at 2026-08-09 09:32:39 on dist-test-slave-7kzw"
server_key: "26e5380779e3a53d0c1c4ada0a79208a"
server_key_iv: "03c765abf25be327bcdc1575c6a38ad9"
server_key_version: "encryptionkey@0"
I20260809 09:32:39.020612 24660 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260809 09:32:39.023187 24684 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 09:32:39.024570 24660 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 09:32:39.024741 24660 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786267941944493-20982-0/minicluster-data/master-0/data,/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786267941944493-20982-0/minicluster-data/master-0/wal
uuid: "26e0fa89ce214d20affe757dee8279a2"
format_stamp: "Formatted at 2026-08-09 09:32:39 on dist-test-slave-7kzw"
server_key: "26e5380779e3a53d0c1c4ada0a79208a"
server_key_iv: "03c765abf25be327bcdc1575c6a38ad9"
server_key_version: "encryptionkey@0"
I20260809 09:32:39.024857 24660 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786267941944493-20982-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786267941944493-20982-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786267941944493-20982-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 09:32:39.050773 24660 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 09:32:39.054318 24660 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 09:32:39.054574 24660 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 09:32:39.062853 24660 rpc_server.cc:307] RPC server started. Bound to: 127.20.125.190:35947
I20260809 09:32:39.062881 24763 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.125.190:35947 every 8 connection(s)
I20260809 09:32:39.063862 24660 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786267941944493-20982-0/minicluster-data/master-0/data/info.pb
I20260809 09:32:39.066783 24765 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 09:32:39.067581 20982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu as pid 24660
I20260809 09:32:39.067670 20982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786267941944493-20982-0/minicluster-data/master-0/wal/instance
I20260809 09:32:39.067975 20982 external_mini_cluster.cc:1428] Setting key 0ccf122d53c98f17263660f020530aa0
I20260809 09:32:39.073068 24765 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 26e0fa89ce214d20affe757dee8279a2: Bootstrap starting.
I20260809 09:32:39.075412 24765 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 26e0fa89ce214d20affe757dee8279a2: Neither blocks nor log segments found. Creating new log.
I20260809 09:32:39.076176 24765 log.cc:824] T 00000000000000000000000000000000 P 26e0fa89ce214d20affe757dee8279a2: Log is configured to *not* fsync() on all Append() calls
Aug 09 09:32:39 dist-test-slave-7kzw krb5kdc[24641](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786267958, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.125.190@KRBTEST.COM
I20260809 09:32:39.078317 24765 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 26e0fa89ce214d20affe757dee8279a2: No bootstrap required, opened a new log
I20260809 09:32:39.083166 24765 raft_consensus.cc:348] T 00000000000000000000000000000000 P 26e0fa89ce214d20affe757dee8279a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26e0fa89ce214d20affe757dee8279a2" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 35947 } }
I20260809 09:32:39.083452 24765 raft_consensus.cc:374] T 00000000000000000000000000000000 P 26e0fa89ce214d20affe757dee8279a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 09:32:39.083544 24765 raft_consensus.cc:729] T 00000000000000000000000000000000 P 26e0fa89ce214d20affe757dee8279a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 26e0fa89ce214d20affe757dee8279a2, State: Initialized, Role: FOLLOWER
I20260809 09:32:39.084012 24772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:39.069461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57692 (local address 127.20.125.190:35947)
0809 09:32:39.069974 (+ 513us) server_negotiation.cc:206] Beginning negotiation
0809 09:32:39.069989 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 09:32:39.070029 (+ 40us) server_negotiation.cc:407] Connection header received
0809 09:32:39.070792 (+ 763us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:32:39.070812 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:32:39.071154 (+ 342us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:32:39.071496 (+ 342us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:32:39.072385 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:39.073770 (+ 1385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:39.074723 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:39.075186 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:39.075378 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:39.077978 (+ 2600us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:32:39.077984 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:32:39.077997 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:32:39.078023 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:32:39.080077 (+ 2054us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:32:39.080664 (+ 587us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:32:39.080670 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:32:39.080676 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:32:39.080830 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:32:39.081377 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:32:39.081381 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:32:39.081383 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:32:39.081926 (+ 543us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:32:39.082143 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0809 09:32:39.082701 (+ 558us) server_negotiation.cc:299] Negotiation successful
0809 09:32:39.083023 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":361,"thread_start_us":141,"threads_started":1}
I20260809 09:32:39.084024 24765 consensus_queue.cc:260] T 00000000000000000000000000000000 P 26e0fa89ce214d20affe757dee8279a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26e0fa89ce214d20affe757dee8279a2" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 35947 } }
I20260809 09:32:39.084314 24765 raft_consensus.cc:388] T 00000000000000000000000000000000 P 26e0fa89ce214d20affe757dee8279a2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 09:32:39.084393 24765 raft_consensus.cc:482] T 00000000000000000000000000000000 P 26e0fa89ce214d20affe757dee8279a2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 09:32:39.084507 24765 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 26e0fa89ce214d20affe757dee8279a2 [term 0 FOLLOWER]: Advancing to term 1
I20260809 09:32:39.085851 24765 raft_consensus.cc:504] T 00000000000000000000000000000000 P 26e0fa89ce214d20affe757dee8279a2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26e0fa89ce214d20affe757dee8279a2" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 35947 } }
I20260809 09:32:39.086251 24765 leader_election.cc:302] T 00000000000000000000000000000000 P 26e0fa89ce214d20affe757dee8279a2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 26e0fa89ce214d20affe757dee8279a2; no voters:
I20260809 09:32:39.086642 24765 leader_election.cc:288] T 00000000000000000000000000000000 P 26e0fa89ce214d20affe757dee8279a2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 09:32:39.087194 24780 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 26e0fa89ce214d20affe757dee8279a2 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 09:32:39.087536 24780 raft_consensus.cc:686] T 00000000000000000000000000000000 P 26e0fa89ce214d20affe757dee8279a2 [term 1 LEADER]: Becoming Leader. State: Replica: 26e0fa89ce214d20affe757dee8279a2, State: Running, Role: LEADER
I20260809 09:32:39.087937 24780 consensus_queue.cc:237] T 00000000000000000000000000000000 P 26e0fa89ce214d20affe757dee8279a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26e0fa89ce214d20affe757dee8279a2" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 35947 } }
I20260809 09:32:39.088076 24765 sys_catalog.cc:564] T 00000000000000000000000000000000 P 26e0fa89ce214d20affe757dee8279a2 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 09:32:39.091066 24777 sys_catalog.cc:455] T 00000000000000000000000000000000 P 26e0fa89ce214d20affe757dee8279a2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 26e0fa89ce214d20affe757dee8279a2. Latest consensus state: current_term: 1 leader_uuid: "26e0fa89ce214d20affe757dee8279a2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26e0fa89ce214d20affe757dee8279a2" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 35947 } } }
I20260809 09:32:39.091110 24782 sys_catalog.cc:455] T 00000000000000000000000000000000 P 26e0fa89ce214d20affe757dee8279a2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "26e0fa89ce214d20affe757dee8279a2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26e0fa89ce214d20affe757dee8279a2" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 35947 } } }
I20260809 09:32:39.091192 24777 sys_catalog.cc:458] T 00000000000000000000000000000000 P 26e0fa89ce214d20affe757dee8279a2 [sys.catalog]: This master's current role is: LEADER
I20260809 09:32:39.091221 24782 sys_catalog.cc:458] T 00000000000000000000000000000000 P 26e0fa89ce214d20affe757dee8279a2 [sys.catalog]: This master's current role is: LEADER
I20260809 09:32:39.091578 24798 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 09:32:39.095588 24798 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 09:32:39.105623 24798 catalog_manager.cc:1347] Generated new cluster ID: af1d286ce83d465981c16c15906404d8
I20260809 09:32:39.105707 24798 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 09:32:39.136263 24798 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 09:32:39.137676 24798 catalog_manager.cc:1504] Loading token signing keys...
I20260809 09:32:39.145336 24798 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 26e0fa89ce214d20affe757dee8279a2: Generated new TSK 0
I20260809 09:32:39.146138 24798 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.20.125.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.125.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.125.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.129.keytab.
Entry for principal kudu/127.20.125.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.129.keytab.
I20260809 09:32:39.189865 20982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
/tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786267941944493-20982-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786267941944493-20982-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786267941944493-20982-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786267941944493-20982-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.125.129:0
--local_ip_for_outbound_sockets=127.20.125.129
--webserver_interface=127.20.125.129
--webserver_port=0
--tserver_master_addrs=127.20.125.190:35947
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.129.keytab
--principal=kudu/127.20.125.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.125.148:37103
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786267941944493-20982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786267941944493-20982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786267941944493-20982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 09:32:39.311185 24817 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 09:32:39.311463 24817 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 09:32:39.311508 24817 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 09:32:39.315531 24817 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 09:32:39.315591 24817 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 09:32:39.315649 24817 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.125.129
I20260809 09:32:39.320837 24817 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.125.148:37103
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786267941944493-20982-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786267941944493-20982-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.129.keytab
--principal=kudu/127.20.125.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.125.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786267941944493-20982-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.125.129
--webserver_port=0
--tserver_master_addrs=127.20.125.190:35947
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24817
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.125.129
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786267941944493-20982-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 08:38:55 UTC on ccycloud.quasar-njjleu.root.comops.site
build id 4575935
I20260809 09:32:39.321869 24817 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 09:32:39.322904 24817 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 09:32:39.332973 24825 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:32:39.333006 24829 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:32:39.332978 24824 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:32:39.333261 24817 server_base.cc:1034] running on GCE node
I20260809 09:32:39.333635 24817 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 09:32:39.334272 24817 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 09:32:39.335493 24817 hybrid_clock.cc:648] HybridClock initialized: now 1786267959335458 us; error 48 us; skew 500 ppm
Aug 09 09:32:39 dist-test-slave-7kzw krb5kdc[24641](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786267959, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 09:32:39.338541 24817 init.cc:377] Logged in from keytab as kudu/127.20.125.129@KRBTEST.COM (short username kudu)
I20260809 09:32:39.339490 24817 webserver.cc:460] Webserver started at http://127.20.125.129:42931/ using document root <none> and password file <none>
I20260809 09:32:39.340061 24817 fs_manager.cc:359] Metadata directory not provided
I20260809 09:32:39.340155 24817 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 09:32:39.340404 24817 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 09:32:39.342418 24817 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786267941944493-20982-0/minicluster-data/ts-0/data/instance:
uuid: "84318b1e23bc47389a5955d5a1592de1"
format_stamp: "Formatted at 2026-08-09 09:32:39 on dist-test-slave-7kzw"
server_key: "8e0eb3405d47bea564b92b7a79682388"
server_key_iv: "d9e9ffa7228115afdd2c924168527e87"
server_key_version: "encryptionkey@0"
I20260809 09:32:39.343055 24817 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786267941944493-20982-0/minicluster-data/ts-0/wal/instance:
uuid: "84318b1e23bc47389a5955d5a1592de1"
format_stamp: "Formatted at 2026-08-09 09:32:39 on dist-test-slave-7kzw"
server_key: "8e0eb3405d47bea564b92b7a79682388"
server_key_iv: "d9e9ffa7228115afdd2c924168527e87"
server_key_version: "encryptionkey@0"
I20260809 09:32:39.346722 24817 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260809 09:32:39.349170 24842 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 09:32:39.350339 24817 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 09:32:39.350526 24817 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786267941944493-20982-0/minicluster-data/ts-0/data,/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786267941944493-20982-0/minicluster-data/ts-0/wal
uuid: "84318b1e23bc47389a5955d5a1592de1"
format_stamp: "Formatted at 2026-08-09 09:32:39 on dist-test-slave-7kzw"
server_key: "8e0eb3405d47bea564b92b7a79682388"
server_key_iv: "d9e9ffa7228115afdd2c924168527e87"
server_key_version: "encryptionkey@0"
I20260809 09:32:39.350648 24817 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786267941944493-20982-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786267941944493-20982-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786267941944493-20982-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 09:32:39.382217 24817 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 09:32:39.385548 24817 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 09:32:39.385769 24817 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 09:32:39.386415 24817 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 09:32:39.387332 24817 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 09:32:39.387406 24817 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 09:32:39.387482 24817 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 09:32:39.387533 24817 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 09:32:39.397226 24817 rpc_server.cc:307] RPC server started. Bound to: 127.20.125.129:46639
I20260809 09:32:39.397249 25028 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.125.129:46639 every 8 connection(s)
I20260809 09:32:39.398205 24817 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786267941944493-20982-0/minicluster-data/ts-0/data/info.pb
Aug 09 09:32:39 dist-test-slave-7kzw krb5kdc[24641](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786267959, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.129@KRBTEST.COM for kudu/127.20.125.190@KRBTEST.COM
I20260809 09:32:39.408413 20982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu as pid 24817
I20260809 09:32:39.408535 20982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786267941944493-20982-0/minicluster-data/ts-0/wal/instance
I20260809 09:32:39.408845 20982 external_mini_cluster.cc:1428] Setting key a424996a776d948f4e930150534209a2
I20260809 09:32:39.412474 24772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:39.400201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.129:43555 (local address 127.20.125.190:35947)
0809 09:32:39.400365 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0809 09:32:39.400369 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 09:32:39.401325 (+ 956us) server_negotiation.cc:407] Connection header received
0809 09:32:39.402742 (+ 1417us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:32:39.402745 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:32:39.402806 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:32:39.402914 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:32:39.404170 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:39.404719 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:39.405852 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:39.406094 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:39.406216 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:39.408804 (+ 2588us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:32:39.408807 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:32:39.408809 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:32:39.408833 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:32:39.410353 (+ 1520us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:32:39.410908 (+ 555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:32:39.410913 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:32:39.410915 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:32:39.411001 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:32:39.411357 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:32:39.411361 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:32:39.411362 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:32:39.411543 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:32:39.411653 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0809 09:32:39.412174 (+ 521us) server_negotiation.cc:299] Negotiation successful
0809 09:32:39.412303 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260809 09:32:39.413255 25034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:39.400579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:35947 (local address 127.20.125.129:43555)
0809 09:32:39.401146 (+ 567us) negotiation.cc:107] Waiting for socket to connect
0809 09:32:39.401199 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0809 09:32:39.402510 (+ 1311us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:32:39.403098 (+ 588us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:32:39.403105 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:32:39.403469 (+ 364us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:32:39.403967 (+ 498us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:32:39.403980 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:32:39.404851 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:32:39.404860 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:32:39.405720 (+ 860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:32:39.405730 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:32:39.406261 (+ 531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:32:39.406265 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:32:39.406428 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:39.407066 (+ 638us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 09:32:39.407083 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:32:39.408617 (+ 1534us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 09:32:39.410503 (+ 1886us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:32:39.410509 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:32:39.410524 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:32:39.410799 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:32:39.411097 (+ 298us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:32:39.411102 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:32:39.411104 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:32:39.411250 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:32:39.411660 (+ 410us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 09:32:39.411669 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 09:32:39.411916 (+ 247us) client_negotiation.cc:769] Sending connection context
0809 09:32:39.412107 (+ 191us) client_negotiation.cc:240] Negotiation successful
0809 09:32:39.412429 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":376,"thread_start_us":118,"threads_started":1}
I20260809 09:32:39.414624 25030 heartbeater.cc:347] Connected to a master server at 127.20.125.190:35947
I20260809 09:32:39.414912 25030 heartbeater.cc:464] Registering TS with master...
I20260809 09:32:39.415382 25030 heartbeater.cc:511] Master 127.20.125.190:35947 requested a full tablet report, sending...
I20260809 09:32:39.417131 24706 ts_manager.cc:194] Registered new tserver with Master: 84318b1e23bc47389a5955d5a1592de1 (127.20.125.129:46639)
I20260809 09:32:39.418843 24706 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.125.129@KRBTEST.COM'} at 127.20.125.129:43555
WARNING: no policy specified for kudu/127.20.125.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.125.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.125.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.130.keytab.
Entry for principal kudu/127.20.125.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.130.keytab.
I20260809 09:32:39.437872 20982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
/tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786267941944493-20982-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786267941944493-20982-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786267941944493-20982-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786267941944493-20982-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.125.130:0
--local_ip_for_outbound_sockets=127.20.125.130
--webserver_interface=127.20.125.130
--webserver_port=0
--tserver_master_addrs=127.20.125.190:35947
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.130.keytab
--principal=kudu/127.20.125.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.125.148:37103
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786267941944493-20982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786267941944493-20982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786267941944493-20982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 09:32:39.552395 25039 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 09:32:39.552655 25039 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 09:32:39.552696 25039 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 09:32:39.556901 25039 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 09:32:39.556957 25039 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 09:32:39.557013 25039 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.125.130
I20260809 09:32:39.562218 25039 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.125.148:37103
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786267941944493-20982-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786267941944493-20982-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.130.keytab
--principal=kudu/127.20.125.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.125.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786267941944493-20982-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.125.130
--webserver_port=0
--tserver_master_addrs=127.20.125.190:35947
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25039
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.125.130
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786267941944493-20982-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 08:38:55 UTC on ccycloud.quasar-njjleu.root.comops.site
build id 4575935
I20260809 09:32:39.563264 25039 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 09:32:39.564502 25039 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 09:32:39.574170 25048 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:32:39.574363 25039 server_base.cc:1034] running on GCE node
W20260809 09:32:39.574165 25054 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:32:39.574165 25049 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:32:39.574980 25039 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 09:32:39.575605 25039 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 09:32:39.576802 25039 hybrid_clock.cc:648] HybridClock initialized: now 1786267959576771 us; error 45 us; skew 500 ppm
Aug 09 09:32:39 dist-test-slave-7kzw krb5kdc[24641](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786267959, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 09:32:39.579655 25039 init.cc:377] Logged in from keytab as kudu/127.20.125.130@KRBTEST.COM (short username kudu)
I20260809 09:32:39.580502 25039 webserver.cc:460] Webserver started at http://127.20.125.130:45591/ using document root <none> and password file <none>
I20260809 09:32:39.581066 25039 fs_manager.cc:359] Metadata directory not provided
I20260809 09:32:39.581162 25039 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 09:32:39.581406 25039 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 09:32:39.583645 25039 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786267941944493-20982-0/minicluster-data/ts-1/data/instance:
uuid: "284d6e7fd6e84710be50a1c713edc335"
format_stamp: "Formatted at 2026-08-09 09:32:39 on dist-test-slave-7kzw"
server_key: "f299349a58a08fb0757b58bf43684e9e"
server_key_iv: "9c2013b952c46bea2873a2c592cc741d"
server_key_version: "encryptionkey@0"
I20260809 09:32:39.584225 25039 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786267941944493-20982-0/minicluster-data/ts-1/wal/instance:
uuid: "284d6e7fd6e84710be50a1c713edc335"
format_stamp: "Formatted at 2026-08-09 09:32:39 on dist-test-slave-7kzw"
server_key: "f299349a58a08fb0757b58bf43684e9e"
server_key_iv: "9c2013b952c46bea2873a2c592cc741d"
server_key_version: "encryptionkey@0"
I20260809 09:32:39.588179 25039 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260809 09:32:39.590732 25064 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 09:32:39.592127 25039 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 09:32:39.592294 25039 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786267941944493-20982-0/minicluster-data/ts-1/data,/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786267941944493-20982-0/minicluster-data/ts-1/wal
uuid: "284d6e7fd6e84710be50a1c713edc335"
format_stamp: "Formatted at 2026-08-09 09:32:39 on dist-test-slave-7kzw"
server_key: "f299349a58a08fb0757b58bf43684e9e"
server_key_iv: "9c2013b952c46bea2873a2c592cc741d"
server_key_version: "encryptionkey@0"
I20260809 09:32:39.592419 25039 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786267941944493-20982-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786267941944493-20982-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786267941944493-20982-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 09:32:39.611716 25039 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 09:32:39.614940 25039 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 09:32:39.615151 25039 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 09:32:39.615770 25039 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 09:32:39.616613 25039 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 09:32:39.616686 25039 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 09:32:39.616760 25039 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 09:32:39.616813 25039 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 09:32:39.625639 25039 rpc_server.cc:307] RPC server started. Bound to: 127.20.125.130:38433
I20260809 09:32:39.625743 25242 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.125.130:38433 every 8 connection(s)
I20260809 09:32:39.626610 25039 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786267941944493-20982-0/minicluster-data/ts-1/data/info.pb
I20260809 09:32:39.635325 20982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu as pid 25039
I20260809 09:32:39.635454 20982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786267941944493-20982-0/minicluster-data/ts-1/wal/instance
I20260809 09:32:39.635753 20982 external_mini_cluster.cc:1428] Setting key d8b31eb0728aa59a5f517295694264b4
Aug 09 09:32:39 dist-test-slave-7kzw krb5kdc[24641](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786267959, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.130@KRBTEST.COM for kudu/127.20.125.190@KRBTEST.COM
I20260809 09:32:39.641156 24772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:39.628524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.130:51639 (local address 127.20.125.190:35947)
0809 09:32:39.628722 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0809 09:32:39.628727 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 09:32:39.630129 (+ 1402us) server_negotiation.cc:407] Connection header received
0809 09:32:39.631031 (+ 902us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:32:39.631034 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:32:39.631091 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:32:39.631173 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:32:39.632259 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:39.632789 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:39.633916 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:39.634154 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:39.634245 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:39.637068 (+ 2823us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:32:39.637072 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:32:39.637074 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:32:39.637101 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:32:39.638672 (+ 1571us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:32:39.639333 (+ 661us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:32:39.639339 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:32:39.639343 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:32:39.639447 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:32:39.639894 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:32:39.639899 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:32:39.639902 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:32:39.640161 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:32:39.640276 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0809 09:32:39.640776 (+ 500us) server_negotiation.cc:299] Negotiation successful
0809 09:32:39.640912 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99}
I20260809 09:32:39.641892 25247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:39.629010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:35947 (local address 127.20.125.130:51639)
0809 09:32:39.629936 (+ 926us) negotiation.cc:107] Waiting for socket to connect
0809 09:32:39.629989 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0809 09:32:39.630831 (+ 842us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:32:39.631298 (+ 467us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:32:39.631304 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:32:39.631657 (+ 353us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:32:39.632109 (+ 452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:32:39.632123 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:32:39.632917 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:32:39.632920 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:32:39.633789 (+ 869us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:32:39.633798 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:32:39.634317 (+ 519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:32:39.634320 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:32:39.634460 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:39.635240 (+ 780us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 09:32:39.635257 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:32:39.636873 (+ 1616us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 09:32:39.638793 (+ 1920us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:32:39.638803 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:32:39.638820 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:32:39.639206 (+ 386us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:32:39.639587 (+ 381us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:32:39.639590 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:32:39.639591 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:32:39.639785 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:32:39.640292 (+ 507us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 09:32:39.640298 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 09:32:39.640528 (+ 230us) client_negotiation.cc:769] Sending connection context
0809 09:32:39.640688 (+ 160us) client_negotiation.cc:240] Negotiation successful
0809 09:32:39.640982 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":707,"thread_start_us":566,"threads_started":1}
I20260809 09:32:39.643157 25244 heartbeater.cc:347] Connected to a master server at 127.20.125.190:35947
I20260809 09:32:39.643446 25244 heartbeater.cc:464] Registering TS with master...
I20260809 09:32:39.644099 25244 heartbeater.cc:511] Master 127.20.125.190:35947 requested a full tablet report, sending...
I20260809 09:32:39.645390 24706 ts_manager.cc:194] Registered new tserver with Master: 284d6e7fd6e84710be50a1c713edc335 (127.20.125.130:38433)
I20260809 09:32:39.646245 24706 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.125.130@KRBTEST.COM'} at 127.20.125.130:51639
WARNING: no policy specified for kudu/127.20.125.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.125.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.125.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.131.keytab.
Entry for principal kudu/127.20.125.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.131.keytab.
I20260809 09:32:39.668108 20982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
/tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786267941944493-20982-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786267941944493-20982-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786267941944493-20982-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786267941944493-20982-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.125.131:0
--local_ip_for_outbound_sockets=127.20.125.131
--webserver_interface=127.20.125.131
--webserver_port=0
--tserver_master_addrs=127.20.125.190:35947
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.131.keytab
--principal=kudu/127.20.125.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.125.148:37103
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786267941944493-20982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786267941944493-20982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786267941944493-20982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 09:32:39.783481 25252 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 09:32:39.783793 25252 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 09:32:39.783875 25252 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 09:32:39.788100 25252 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 09:32:39.788211 25252 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 09:32:39.788308 25252 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.125.131
I20260809 09:32:39.794099 25252 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.125.148:37103
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786267941944493-20982-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786267941944493-20982-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.131.keytab
--principal=kudu/127.20.125.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.125.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786267941944493-20982-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.125.131
--webserver_port=0
--tserver_master_addrs=127.20.125.190:35947
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25252
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.125.131
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786267941944493-20982-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 08:38:55 UTC on ccycloud.quasar-njjleu.root.comops.site
build id 4575935
I20260809 09:32:39.795429 25252 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 09:32:39.796603 25252 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 09:32:39.806670 25264 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:32:39.806699 25252 server_base.cc:1034] running on GCE node
W20260809 09:32:39.806667 25262 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:32:39.806667 25261 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:32:39.807472 25252 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 09:32:39.808136 25252 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 09:32:39.809374 25252 hybrid_clock.cc:648] HybridClock initialized: now 1786267959809337 us; error 48 us; skew 500 ppm
Aug 09 09:32:39 dist-test-slave-7kzw krb5kdc[24641](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786267959, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 09:32:39.812764 25252 init.cc:377] Logged in from keytab as kudu/127.20.125.131@KRBTEST.COM (short username kudu)
I20260809 09:32:39.813666 25252 webserver.cc:460] Webserver started at http://127.20.125.131:34651/ using document root <none> and password file <none>
I20260809 09:32:39.814276 25252 fs_manager.cc:359] Metadata directory not provided
I20260809 09:32:39.814370 25252 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 09:32:39.814640 25252 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 09:32:39.816850 25252 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786267941944493-20982-0/minicluster-data/ts-2/data/instance:
uuid: "33bbcde679dd4d6eb083829e63c75e89"
format_stamp: "Formatted at 2026-08-09 09:32:39 on dist-test-slave-7kzw"
server_key: "c6607f70ae9fe9301b7b7b686e07fb97"
server_key_iv: "49e503e740382178eab385949a1f8787"
server_key_version: "encryptionkey@0"
I20260809 09:32:39.817437 25252 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786267941944493-20982-0/minicluster-data/ts-2/wal/instance:
uuid: "33bbcde679dd4d6eb083829e63c75e89"
format_stamp: "Formatted at 2026-08-09 09:32:39 on dist-test-slave-7kzw"
server_key: "c6607f70ae9fe9301b7b7b686e07fb97"
server_key_iv: "49e503e740382178eab385949a1f8787"
server_key_version: "encryptionkey@0"
I20260809 09:32:39.821458 25252 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260809 09:32:39.824250 25278 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 09:32:39.825553 25252 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.001s
I20260809 09:32:39.825716 25252 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786267941944493-20982-0/minicluster-data/ts-2/data,/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786267941944493-20982-0/minicluster-data/ts-2/wal
uuid: "33bbcde679dd4d6eb083829e63c75e89"
format_stamp: "Formatted at 2026-08-09 09:32:39 on dist-test-slave-7kzw"
server_key: "c6607f70ae9fe9301b7b7b686e07fb97"
server_key_iv: "49e503e740382178eab385949a1f8787"
server_key_version: "encryptionkey@0"
I20260809 09:32:39.825839 25252 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786267941944493-20982-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786267941944493-20982-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786267941944493-20982-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 09:32:39.841753 25252 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 09:32:39.845029 25252 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 09:32:39.845255 25252 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 09:32:39.845839 25252 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 09:32:39.846774 25252 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 09:32:39.846843 25252 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 09:32:39.846917 25252 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 09:32:39.846971 25252 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 09:32:39.856256 25252 rpc_server.cc:307] RPC server started. Bound to: 127.20.125.131:33243
I20260809 09:32:39.856274 25455 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.125.131:33243 every 8 connection(s)
I20260809 09:32:39.857206 25252 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786267941944493-20982-0/minicluster-data/ts-2/data/info.pb
I20260809 09:32:39.866008 20982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu as pid 25252
I20260809 09:32:39.866140 20982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786267941944493-20982-0/minicluster-data/ts-2/wal/instance
I20260809 09:32:39.866456 20982 external_mini_cluster.cc:1428] Setting key ec4a555a84b5c31a31515142442dd1bd
Aug 09 09:32:39 dist-test-slave-7kzw krb5kdc[24641](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786267959, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.131@KRBTEST.COM for kudu/127.20.125.190@KRBTEST.COM
I20260809 09:32:39.872728 24772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:39.859381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.131:51919 (local address 127.20.125.190:35947)
0809 09:32:39.859624 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0809 09:32:39.859629 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 09:32:39.860860 (+ 1231us) server_negotiation.cc:407] Connection header received
0809 09:32:39.861689 (+ 829us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:32:39.861692 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:32:39.861746 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:32:39.861823 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:32:39.863179 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:39.863739 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:39.865123 (+ 1384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:39.865364 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:39.865450 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:39.868577 (+ 3127us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:32:39.868581 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:32:39.868584 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:32:39.868612 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:32:39.870299 (+ 1687us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:32:39.870993 (+ 694us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:32:39.870996 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:32:39.870997 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:32:39.871092 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:32:39.871524 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:32:39.871527 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:32:39.871528 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:32:39.871762 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:32:39.871846 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0809 09:32:39.872429 (+ 583us) server_negotiation.cc:299] Negotiation successful
0809 09:32:39.872548 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260809 09:32:39.873538 25458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:39.859873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:35947 (local address 127.20.125.131:51919)
0809 09:32:39.860693 (+ 820us) negotiation.cc:107] Waiting for socket to connect
0809 09:32:39.860739 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 09:32:39.861495 (+ 756us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:32:39.861997 (+ 502us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:32:39.862004 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:32:39.862394 (+ 390us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:32:39.863017 (+ 623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:32:39.863031 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:32:39.863889 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:32:39.863893 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:32:39.864960 (+ 1067us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:32:39.864972 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:32:39.865512 (+ 540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:32:39.865515 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:32:39.865672 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:39.866381 (+ 709us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 09:32:39.866399 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:32:39.868409 (+ 2010us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 09:32:39.870438 (+ 2029us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:32:39.870443 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:32:39.870457 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:32:39.870859 (+ 402us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:32:39.871213 (+ 354us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:32:39.871216 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:32:39.871217 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:32:39.871371 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:32:39.871874 (+ 503us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 09:32:39.871880 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 09:32:39.872135 (+ 255us) client_negotiation.cc:769] Sending connection context
0809 09:32:39.872305 (+ 170us) client_negotiation.cc:240] Negotiation successful
0809 09:32:39.872576 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":585,"thread_start_us":155,"threads_started":1}
I20260809 09:32:39.875104 25456 heartbeater.cc:347] Connected to a master server at 127.20.125.190:35947
I20260809 09:32:39.875406 25456 heartbeater.cc:464] Registering TS with master...
I20260809 09:32:39.876060 25456 heartbeater.cc:511] Master 127.20.125.190:35947 requested a full tablet report, sending...
I20260809 09:32:39.877279 24706 ts_manager.cc:194] Registered new tserver with Master: 33bbcde679dd4d6eb083829e63c75e89 (127.20.125.131:33243)
I20260809 09:32:39.878031 24706 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.125.131@KRBTEST.COM'} at 127.20.125.131:51919
I20260809 09:32:39.881196 20982 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260809 09:32:39.899396 24772 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 09:32:39.897403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57702 (local address 127.20.125.190:35947)
0809 09:32:39.897565 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0809 09:32:39.897571 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 09:32:39.897863 (+ 292us) server_negotiation.cc:407] Connection header received
0809 09:32:39.898042 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:32:39.898046 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:32:39.898105 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:32:39.898196 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:32:39.899207 (+ 1011us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57702: BlockingRecv error: recv got EOF from 127.0.0.1:57702 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 09:32:39.900699 20982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu with pid 24817
I20260809 09:32:39.910696 20982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu with pid 25039
I20260809 09:32:39.927774 20982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu with pid 25252
I20260809 09:32:39.942672 20982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu with pid 24660
2026-08-09T09:32:39Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (2440 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786267941944493-20982-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 09:32:40 dist-test-slave-7kzw krb5kdc[25470](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 09:32:40 dist-test-slave-7kzw krb5kdc[25470](info): set up 2 sockets
Aug 09 09:32:40 dist-test-slave-7kzw krb5kdc[25470](info): commencing operation
krb5kdc: starting...
W20260809 09:32:41.176643 20982 mini_kdc.cc:121] Time spent starting KDC: real 1.207s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786267941944493-20982-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786267941944493-20982-0/krb5kdc/test-user.keytab.
Aug 09 09:32:41 dist-test-slave-7kzw krb5kdc[25470](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786267961, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T09:32:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T09:32:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.125.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.125.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.125.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.190.keytab.
Entry for principal kudu/127.20.125.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.190.keytab.
I20260809 09:32:41.318974 20982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
/tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786267941944493-20982-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786267941944493-20982-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786267941944493-20982-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786267941944493-20982-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.125.190:42011
--webserver_interface=127.20.125.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.190.keytab
--principal=kudu/127.20.125.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.125.148:39081
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.125.190:42011
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786267941944493-20982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786267941944493-20982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786267941944493-20982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 09:32:41.434649 25491 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 09:32:41.434996 25491 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 09:32:41.435073 25491 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 09:32:41.439112 25491 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 09:32:41.439217 25491 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 09:32:41.439254 25491 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 09:32:41.439292 25491 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 09:32:41.439322 25491 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 09:32:41.444715 25491 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.125.148:39081
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786267941944493-20982-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786267941944493-20982-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.125.190:42011
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.190.keytab
--principal=kudu/127.20.125.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.125.190:42011
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786267941944493-20982-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.125.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25491
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786267941944493-20982-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 08:38:55 UTC on ccycloud.quasar-njjleu.root.comops.site
build id 4575935
I20260809 09:32:41.445914 25491 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 09:32:41.447044 25491 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 09:32:41.455937 25499 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:32:41.456065 25491 server_base.cc:1034] running on GCE node
W20260809 09:32:41.455929 25498 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:32:41.455901 25504 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:32:41.456846 25491 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 09:32:41.458086 25491 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 09:32:41.459280 25491 hybrid_clock.cc:648] HybridClock initialized: now 1786267961459238 us; error 56 us; skew 500 ppm
Aug 09 09:32:41 dist-test-slave-7kzw krb5kdc[25470](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786267961, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 09:32:41.467183 25491 init.cc:377] Logged in from keytab as kudu/127.20.125.190@KRBTEST.COM (short username kudu)
I20260809 09:32:41.468178 25491 webserver.cc:460] Webserver started at http://127.20.125.190:45189/ using document root <none> and password file <none>
I20260809 09:32:41.468978 25491 fs_manager.cc:359] Metadata directory not provided
I20260809 09:32:41.469079 25491 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 09:32:41.469321 25491 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 09:32:41.471491 25491 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786267941944493-20982-0/minicluster-data/master-0/data/instance:
uuid: "cacdf7a410bb436eb238fe52078265f9"
format_stamp: "Formatted at 2026-08-09 09:32:41 on dist-test-slave-7kzw"
server_key: "e4530736942cb9b6d2d5aa560cf5cc3b"
server_key_iv: "9f5d12180ae96b8c94e0059b661d9410"
server_key_version: "encryptionkey@0"
I20260809 09:32:41.472088 25491 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786267941944493-20982-0/minicluster-data/master-0/wal/instance:
uuid: "cacdf7a410bb436eb238fe52078265f9"
format_stamp: "Formatted at 2026-08-09 09:32:41 on dist-test-slave-7kzw"
server_key: "e4530736942cb9b6d2d5aa560cf5cc3b"
server_key_iv: "9f5d12180ae96b8c94e0059b661d9410"
server_key_version: "encryptionkey@0"
I20260809 09:32:41.476016 25491 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260809 09:32:41.478441 25512 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 09:32:41.479841 25491 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 09:32:41.479991 25491 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786267941944493-20982-0/minicluster-data/master-0/data,/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786267941944493-20982-0/minicluster-data/master-0/wal
uuid: "cacdf7a410bb436eb238fe52078265f9"
format_stamp: "Formatted at 2026-08-09 09:32:41 on dist-test-slave-7kzw"
server_key: "e4530736942cb9b6d2d5aa560cf5cc3b"
server_key_iv: "9f5d12180ae96b8c94e0059b661d9410"
server_key_version: "encryptionkey@0"
I20260809 09:32:41.480111 25491 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786267941944493-20982-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786267941944493-20982-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786267941944493-20982-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 09:32:41.504345 25491 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 09:32:41.507881 25491 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 09:32:41.508126 25491 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 09:32:41.515844 25491 rpc_server.cc:307] RPC server started. Bound to: 127.20.125.190:42011
I20260809 09:32:41.516163 25584 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.125.190:42011 every 8 connection(s)
I20260809 09:32:41.517069 25491 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786267941944493-20982-0/minicluster-data/master-0/data/info.pb
I20260809 09:32:41.519964 25585 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 09:32:41.525640 25585 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cacdf7a410bb436eb238fe52078265f9: Bootstrap starting.
I20260809 09:32:41.526863 20982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu as pid 25491
I20260809 09:32:41.526986 20982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786267941944493-20982-0/minicluster-data/master-0/wal/instance
I20260809 09:32:41.527316 20982 external_mini_cluster.cc:1428] Setting key ce792d1cbe06939cf8ff807c26dfe611
I20260809 09:32:41.528756 25585 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cacdf7a410bb436eb238fe52078265f9: Neither blocks nor log segments found. Creating new log.
I20260809 09:32:41.529541 25585 log.cc:824] T 00000000000000000000000000000000 P cacdf7a410bb436eb238fe52078265f9: Log is configured to *not* fsync() on all Append() calls
I20260809 09:32:41.531747 25585 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cacdf7a410bb436eb238fe52078265f9: No bootstrap required, opened a new log
I20260809 09:32:41.535573 25585 raft_consensus.cc:348] T 00000000000000000000000000000000 P cacdf7a410bb436eb238fe52078265f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cacdf7a410bb436eb238fe52078265f9" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 42011 } }
I20260809 09:32:41.535849 25585 raft_consensus.cc:374] T 00000000000000000000000000000000 P cacdf7a410bb436eb238fe52078265f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 09:32:41.535923 25585 raft_consensus.cc:729] T 00000000000000000000000000000000 P cacdf7a410bb436eb238fe52078265f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cacdf7a410bb436eb238fe52078265f9, State: Initialized, Role: FOLLOWER
Aug 09 09:32:41 dist-test-slave-7kzw krb5kdc[25470](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786267961, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.125.190@KRBTEST.COM
I20260809 09:32:41.536439 25585 consensus_queue.cc:260] T 00000000000000000000000000000000 P cacdf7a410bb436eb238fe52078265f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cacdf7a410bb436eb238fe52078265f9" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 42011 } }
I20260809 09:32:41.536617 25585 raft_consensus.cc:388] T 00000000000000000000000000000000 P cacdf7a410bb436eb238fe52078265f9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 09:32:41.536703 25585 raft_consensus.cc:482] T 00000000000000000000000000000000 P cacdf7a410bb436eb238fe52078265f9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 09:32:41.536823 25585 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cacdf7a410bb436eb238fe52078265f9 [term 0 FOLLOWER]: Advancing to term 1
I20260809 09:32:41.538017 25585 raft_consensus.cc:504] T 00000000000000000000000000000000 P cacdf7a410bb436eb238fe52078265f9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cacdf7a410bb436eb238fe52078265f9" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 42011 } }
I20260809 09:32:41.538455 25585 leader_election.cc:302] T 00000000000000000000000000000000 P cacdf7a410bb436eb238fe52078265f9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cacdf7a410bb436eb238fe52078265f9; no voters:
I20260809 09:32:41.538885 25585 leader_election.cc:288] T 00000000000000000000000000000000 P cacdf7a410bb436eb238fe52078265f9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 09:32:41.539009 25594 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cacdf7a410bb436eb238fe52078265f9 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 09:32:41.539249 25594 raft_consensus.cc:686] T 00000000000000000000000000000000 P cacdf7a410bb436eb238fe52078265f9 [term 1 LEADER]: Becoming Leader. State: Replica: cacdf7a410bb436eb238fe52078265f9, State: Running, Role: LEADER
I20260809 09:32:41.539671 25594 consensus_queue.cc:237] T 00000000000000000000000000000000 P cacdf7a410bb436eb238fe52078265f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cacdf7a410bb436eb238fe52078265f9" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 42011 } }
I20260809 09:32:41.540352 25585 sys_catalog.cc:564] T 00000000000000000000000000000000 P cacdf7a410bb436eb238fe52078265f9 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 09:32:41.541476 25597 sys_catalog.cc:455] T 00000000000000000000000000000000 P cacdf7a410bb436eb238fe52078265f9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cacdf7a410bb436eb238fe52078265f9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cacdf7a410bb436eb238fe52078265f9" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 42011 } } }
I20260809 09:32:41.541631 25597 sys_catalog.cc:458] T 00000000000000000000000000000000 P cacdf7a410bb436eb238fe52078265f9 [sys.catalog]: This master's current role is: LEADER
I20260809 09:32:41.541981 25599 sys_catalog.cc:455] T 00000000000000000000000000000000 P cacdf7a410bb436eb238fe52078265f9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cacdf7a410bb436eb238fe52078265f9. Latest consensus state: current_term: 1 leader_uuid: "cacdf7a410bb436eb238fe52078265f9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cacdf7a410bb436eb238fe52078265f9" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 42011 } } }
I20260809 09:32:41.542132 25599 sys_catalog.cc:458] T 00000000000000000000000000000000 P cacdf7a410bb436eb238fe52078265f9 [sys.catalog]: This master's current role is: LEADER
I20260809 09:32:41.543926 25590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:41.528510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58496 (local address 127.20.125.190:42011)
0809 09:32:41.529064 (+ 554us) server_negotiation.cc:206] Beginning negotiation
0809 09:32:41.529080 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 09:32:41.529117 (+ 37us) server_negotiation.cc:407] Connection header received
0809 09:32:41.529836 (+ 719us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:32:41.529856 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:32:41.530320 (+ 464us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:32:41.530751 (+ 431us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:32:41.531620 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:41.532680 (+ 1060us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:41.533573 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:41.534055 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:41.534248 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:41.537131 (+ 2883us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:32:41.537136 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:32:41.537149 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:32:41.537177 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:32:41.539468 (+ 2291us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:32:41.540058 (+ 590us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:32:41.540064 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:32:41.540070 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:32:41.540228 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:32:41.540605 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:32:41.540609 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:32:41.540611 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:32:41.541187 (+ 576us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:32:41.542363 (+ 1176us) server_negotiation.cc:1035] Waiting for connection context
0809 09:32:41.542563 (+ 200us) server_negotiation.cc:299] Negotiation successful
0809 09:32:41.542998 (+ 435us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":391,"thread_start_us":157,"threads_started":1}
W20260809 09:32:41.545346 25619 catalog_manager.cc:1558] T 00000000000000000000000000000000 P cacdf7a410bb436eb238fe52078265f9: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 09:32:41.545451 25619 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 09:32:41.545552 25610 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 09:32:41.546568 25610 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 09:32:41.547084 20982 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 09:32:41.558883 25610 catalog_manager.cc:1347] Generated new cluster ID: 92156bb6c3544395888ed7ac803d7c53
I20260809 09:32:41.558984 25610 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 09:32:41.572283 25590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:41.566446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58504 (local address 127.20.125.190:42011)
0809 09:32:41.566574 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0809 09:32:41.566578 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 09:32:41.566867 (+ 289us) server_negotiation.cc:407] Connection header received
0809 09:32:41.566991 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:32:41.566994 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:32:41.567061 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:32:41.567155 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:32:41.567744 (+ 589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:41.568567 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:41.569736 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:41.570104 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:41.570227 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:41.571168 (+ 941us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:32:41.571172 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:32:41.571175 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 09:32:41.571770 (+ 595us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:32:41.571803 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 09:32:41.571823 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:32:41.571914 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0809 09:32:41.572057 (+ 143us) server_negotiation.cc:299] Negotiation successful
0809 09:32:41.572120 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 09:32:41.573632 25610 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 09:32:41.574678 25610 catalog_manager.cc:1504] Loading token signing keys...
I20260809 09:32:41.587284 25610 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cacdf7a410bb436eb238fe52078265f9: Generated new TSK 0
I20260809 09:32:41.587967 25610 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 09:32:41.599287 20982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu with pid 25491
2026-08-09T09:32:41Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1655 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786267941944493-20982-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 09:32:41 dist-test-slave-7kzw krb5kdc[25643](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 09:32:41 dist-test-slave-7kzw krb5kdc[25643](info): set up 2 sockets
Aug 09 09:32:41 dist-test-slave-7kzw krb5kdc[25643](info): commencing operation
krb5kdc: starting...
W20260809 09:32:42.836334 20982 mini_kdc.cc:121] Time spent starting KDC: real 1.211s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786267941944493-20982-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786267941944493-20982-0/krb5kdc/test-user.keytab.
Aug 09 09:32:42 dist-test-slave-7kzw krb5kdc[25643](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786267962, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T09:32:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T09:32:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.125.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.125.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.125.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.190.keytab.
Entry for principal kudu/127.20.125.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.190.keytab.
I20260809 09:32:42.977306 20982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
/tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786267941944493-20982-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786267941944493-20982-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786267941944493-20982-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786267941944493-20982-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.125.190:44051
--webserver_interface=127.20.125.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.190.keytab
--principal=kudu/127.20.125.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.125.148:44507
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.125.190:44051
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786267941944493-20982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786267941944493-20982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786267941944493-20982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 09:32:43.094547 25661 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 09:32:43.094813 25661 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 09:32:43.094856 25661 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 09:32:43.098783 25661 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 09:32:43.098834 25661 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 09:32:43.098852 25661 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 09:32:43.098868 25661 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 09:32:43.098883 25661 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 09:32:43.104050 25661 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.125.148:44507
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786267941944493-20982-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786267941944493-20982-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.125.190:44051
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.190.keytab
--principal=kudu/127.20.125.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.125.190:44051
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786267941944493-20982-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.125.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25661
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786267941944493-20982-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 08:38:55 UTC on ccycloud.quasar-njjleu.root.comops.site
build id 4575935
I20260809 09:32:43.105057 25661 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 09:32:43.106027 25661 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 09:32:43.114547 25667 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:32:43.114576 25670 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:32:43.114533 25668 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:32:43.115281 25661 server_base.cc:1034] running on GCE node
I20260809 09:32:43.115897 25661 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 09:32:43.117105 25661 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 09:32:43.118319 25661 hybrid_clock.cc:648] HybridClock initialized: now 1786267963118298 us; error 32 us; skew 500 ppm
Aug 09 09:32:43 dist-test-slave-7kzw krb5kdc[25643](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786267963, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 09:32:43.121430 25661 init.cc:377] Logged in from keytab as kudu/127.20.125.190@KRBTEST.COM (short username kudu)
I20260809 09:32:43.122534 25661 webserver.cc:460] Webserver started at http://127.20.125.190:37083/ using document root <none> and password file <none>
I20260809 09:32:43.123153 25661 fs_manager.cc:359] Metadata directory not provided
I20260809 09:32:43.123256 25661 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 09:32:43.123512 25661 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 09:32:43.125829 25661 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786267941944493-20982-0/minicluster-data/master-0/data/instance:
uuid: "b79f32f745af441e987c24eb9e60ab72"
format_stamp: "Formatted at 2026-08-09 09:32:43 on dist-test-slave-7kzw"
server_key: "ef171ddca3a002a82e784962da8b4c9e"
server_key_iv: "fc4a472b431b1925615187664a8094c2"
server_key_version: "encryptionkey@0"
I20260809 09:32:43.126562 25661 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786267941944493-20982-0/minicluster-data/master-0/wal/instance:
uuid: "b79f32f745af441e987c24eb9e60ab72"
format_stamp: "Formatted at 2026-08-09 09:32:43 on dist-test-slave-7kzw"
server_key: "ef171ddca3a002a82e784962da8b4c9e"
server_key_iv: "fc4a472b431b1925615187664a8094c2"
server_key_version: "encryptionkey@0"
I20260809 09:32:43.131994 25661 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 09:32:43.138453 25678 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 09:32:43.140280 25661 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.007s sys 0.000s
I20260809 09:32:43.140429 25661 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786267941944493-20982-0/minicluster-data/master-0/data,/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786267941944493-20982-0/minicluster-data/master-0/wal
uuid: "b79f32f745af441e987c24eb9e60ab72"
format_stamp: "Formatted at 2026-08-09 09:32:43 on dist-test-slave-7kzw"
server_key: "ef171ddca3a002a82e784962da8b4c9e"
server_key_iv: "fc4a472b431b1925615187664a8094c2"
server_key_version: "encryptionkey@0"
I20260809 09:32:43.140631 25661 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786267941944493-20982-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786267941944493-20982-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786267941944493-20982-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 09:32:43.164513 25661 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 09:32:43.167826 25661 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 09:32:43.167997 25661 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 09:32:43.175464 25661 rpc_server.cc:307] RPC server started. Bound to: 127.20.125.190:44051
I20260809 09:32:43.175823 25763 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.125.190:44051 every 8 connection(s)
I20260809 09:32:43.176515 25661 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786267941944493-20982-0/minicluster-data/master-0/data/info.pb
I20260809 09:32:43.179674 25764 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 09:32:43.184818 20982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu as pid 25661
I20260809 09:32:43.184978 20982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786267941944493-20982-0/minicluster-data/master-0/wal/instance
I20260809 09:32:43.185294 20982 external_mini_cluster.cc:1428] Setting key c53d37f6898a288204526348f0a166b4
I20260809 09:32:43.186129 25764 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b79f32f745af441e987c24eb9e60ab72: Bootstrap starting.
I20260809 09:32:43.188733 25764 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b79f32f745af441e987c24eb9e60ab72: Neither blocks nor log segments found. Creating new log.
I20260809 09:32:43.189733 25764 log.cc:824] T 00000000000000000000000000000000 P b79f32f745af441e987c24eb9e60ab72: Log is configured to *not* fsync() on all Append() calls
I20260809 09:32:43.192184 25764 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b79f32f745af441e987c24eb9e60ab72: No bootstrap required, opened a new log
Aug 09 09:32:43 dist-test-slave-7kzw krb5kdc[25643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786267962, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.125.190@KRBTEST.COM
I20260809 09:32:43.197381 25764 raft_consensus.cc:348] T 00000000000000000000000000000000 P b79f32f745af441e987c24eb9e60ab72 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b79f32f745af441e987c24eb9e60ab72" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 44051 } }
I20260809 09:32:43.197656 25764 raft_consensus.cc:374] T 00000000000000000000000000000000 P b79f32f745af441e987c24eb9e60ab72 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 09:32:43.197751 25764 raft_consensus.cc:729] T 00000000000000000000000000000000 P b79f32f745af441e987c24eb9e60ab72 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b79f32f745af441e987c24eb9e60ab72, State: Initialized, Role: FOLLOWER
I20260809 09:32:43.198261 25764 consensus_queue.cc:260] T 00000000000000000000000000000000 P b79f32f745af441e987c24eb9e60ab72 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b79f32f745af441e987c24eb9e60ab72" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 44051 } }
I20260809 09:32:43.198416 25764 raft_consensus.cc:388] T 00000000000000000000000000000000 P b79f32f745af441e987c24eb9e60ab72 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 09:32:43.198535 25764 raft_consensus.cc:482] T 00000000000000000000000000000000 P b79f32f745af441e987c24eb9e60ab72 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 09:32:43.198670 25764 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b79f32f745af441e987c24eb9e60ab72 [term 0 FOLLOWER]: Advancing to term 1
I20260809 09:32:43.199644 25764 raft_consensus.cc:504] T 00000000000000000000000000000000 P b79f32f745af441e987c24eb9e60ab72 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b79f32f745af441e987c24eb9e60ab72" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 44051 } }
I20260809 09:32:43.200019 25764 leader_election.cc:302] T 00000000000000000000000000000000 P b79f32f745af441e987c24eb9e60ab72 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b79f32f745af441e987c24eb9e60ab72; no voters:
I20260809 09:32:43.200385 25764 leader_election.cc:288] T 00000000000000000000000000000000 P b79f32f745af441e987c24eb9e60ab72 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 09:32:43.200477 25770 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b79f32f745af441e987c24eb9e60ab72 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 09:32:43.200717 25770 raft_consensus.cc:686] T 00000000000000000000000000000000 P b79f32f745af441e987c24eb9e60ab72 [term 1 LEADER]: Becoming Leader. State: Replica: b79f32f745af441e987c24eb9e60ab72, State: Running, Role: LEADER
I20260809 09:32:43.201058 25770 consensus_queue.cc:237] T 00000000000000000000000000000000 P b79f32f745af441e987c24eb9e60ab72 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b79f32f745af441e987c24eb9e60ab72" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 44051 } }
I20260809 09:32:43.201680 25768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:43.186843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58212 (local address 127.20.125.190:44051)
0809 09:32:43.187361 (+ 518us) server_negotiation.cc:206] Beginning negotiation
0809 09:32:43.187376 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 09:32:43.187413 (+ 37us) server_negotiation.cc:407] Connection header received
0809 09:32:43.188078 (+ 665us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:32:43.188097 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:32:43.188442 (+ 345us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:32:43.188724 (+ 282us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:32:43.189686 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:43.190966 (+ 1280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:43.192231 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:43.192988 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:43.193159 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:43.196196 (+ 3037us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:32:43.196201 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:32:43.196217 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:32:43.196247 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:32:43.198464 (+ 2217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:32:43.199024 (+ 560us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:32:43.199035 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:32:43.199040 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:32:43.199164 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:32:43.199458 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:32:43.199462 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:32:43.199463 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:32:43.199939 (+ 476us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:32:43.200148 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0809 09:32:43.200447 (+ 299us) server_negotiation.cc:299] Negotiation successful
0809 09:32:43.200798 (+ 351us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":338,"thread_start_us":160,"threads_started":1}
I20260809 09:32:43.201887 25764 sys_catalog.cc:564] T 00000000000000000000000000000000 P b79f32f745af441e987c24eb9e60ab72 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 09:32:43.202327 25770 sys_catalog.cc:455] T 00000000000000000000000000000000 P b79f32f745af441e987c24eb9e60ab72 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b79f32f745af441e987c24eb9e60ab72" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b79f32f745af441e987c24eb9e60ab72" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 44051 } } }
I20260809 09:32:43.202450 25770 sys_catalog.cc:458] T 00000000000000000000000000000000 P b79f32f745af441e987c24eb9e60ab72 [sys.catalog]: This master's current role is: LEADER
I20260809 09:32:43.204053 25770 sys_catalog.cc:455] T 00000000000000000000000000000000 P b79f32f745af441e987c24eb9e60ab72 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b79f32f745af441e987c24eb9e60ab72. Latest consensus state: current_term: 1 leader_uuid: "b79f32f745af441e987c24eb9e60ab72" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b79f32f745af441e987c24eb9e60ab72" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 44051 } } }
I20260809 09:32:43.204239 25770 sys_catalog.cc:458] T 00000000000000000000000000000000 P b79f32f745af441e987c24eb9e60ab72 [sys.catalog]: This master's current role is: LEADER
W20260809 09:32:43.207291 25792 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b79f32f745af441e987c24eb9e60ab72: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 09:32:43.207386 25792 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 09:32:43.207458 25777 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 09:32:43.208407 25777 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 09:32:43.214123 25777 catalog_manager.cc:1347] Generated new cluster ID: 4826555e9a4345ebbc36ef6082203267
I20260809 09:32:43.214195 25777 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 09:32:43.231137 25777 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 09:32:43.232144 25777 catalog_manager.cc:1504] Loading token signing keys...
I20260809 09:32:43.242051 25777 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b79f32f745af441e987c24eb9e60ab72: Generated new TSK 0
I20260809 09:32:43.242802 25777 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.20.125.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.125.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.125.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.129.keytab.
Entry for principal kudu/127.20.125.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.129.keytab.
I20260809 09:32:43.279791 20982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
/tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786267941944493-20982-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786267941944493-20982-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786267941944493-20982-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786267941944493-20982-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.125.129:0
--local_ip_for_outbound_sockets=127.20.125.129
--webserver_interface=127.20.125.129
--webserver_port=0
--tserver_master_addrs=127.20.125.190:44051
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.129.keytab
--principal=kudu/127.20.125.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.125.148:44507
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786267941944493-20982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786267941944493-20982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786267941944493-20982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 09:32:43.394586 25801 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 09:32:43.394845 25801 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 09:32:43.394889 25801 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 09:32:43.398876 25801 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 09:32:43.398933 25801 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 09:32:43.398988 25801 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.125.129
I20260809 09:32:43.404407 25801 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.125.148:44507
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786267941944493-20982-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786267941944493-20982-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.129.keytab
--principal=kudu/127.20.125.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.125.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786267941944493-20982-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.125.129
--webserver_port=0
--tserver_master_addrs=127.20.125.190:44051
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25801
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.125.129
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786267941944493-20982-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 08:38:55 UTC on ccycloud.quasar-njjleu.root.comops.site
build id 4575935
I20260809 09:32:43.405447 25801 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 09:32:43.406535 25801 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 09:32:43.416388 25808 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:32:43.416381 25807 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:32:43.416651 25801 server_base.cc:1034] running on GCE node
W20260809 09:32:43.416365 25812 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:32:43.417174 25801 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 09:32:43.417856 25801 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 09:32:43.419040 25801 hybrid_clock.cc:648] HybridClock initialized: now 1786267963419005 us; error 47 us; skew 500 ppm
Aug 09 09:32:43 dist-test-slave-7kzw krb5kdc[25643](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786267963, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 09:32:43.421908 25801 init.cc:377] Logged in from keytab as kudu/127.20.125.129@KRBTEST.COM (short username kudu)
I20260809 09:32:43.422808 25801 webserver.cc:460] Webserver started at http://127.20.125.129:34337/ using document root <none> and password file <none>
I20260809 09:32:43.423370 25801 fs_manager.cc:359] Metadata directory not provided
I20260809 09:32:43.423461 25801 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 09:32:43.423689 25801 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 09:32:43.425808 25801 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786267941944493-20982-0/minicluster-data/ts-0/data/instance:
uuid: "b702e81224404739b778e70e45663a28"
format_stamp: "Formatted at 2026-08-09 09:32:43 on dist-test-slave-7kzw"
server_key: "357cd5ac6c11f8e66237f1fd3981cb16"
server_key_iv: "8a241f00c60f69dcb38345ce88c21acd"
server_key_version: "encryptionkey@0"
I20260809 09:32:43.426379 25801 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786267941944493-20982-0/minicluster-data/ts-0/wal/instance:
uuid: "b702e81224404739b778e70e45663a28"
format_stamp: "Formatted at 2026-08-09 09:32:43 on dist-test-slave-7kzw"
server_key: "357cd5ac6c11f8e66237f1fd3981cb16"
server_key_iv: "8a241f00c60f69dcb38345ce88c21acd"
server_key_version: "encryptionkey@0"
I20260809 09:32:43.430059 25801 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260809 09:32:43.432556 25820 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 09:32:43.433899 25801 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 09:32:43.434062 25801 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786267941944493-20982-0/minicluster-data/ts-0/data,/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786267941944493-20982-0/minicluster-data/ts-0/wal
uuid: "b702e81224404739b778e70e45663a28"
format_stamp: "Formatted at 2026-08-09 09:32:43 on dist-test-slave-7kzw"
server_key: "357cd5ac6c11f8e66237f1fd3981cb16"
server_key_iv: "8a241f00c60f69dcb38345ce88c21acd"
server_key_version: "encryptionkey@0"
I20260809 09:32:43.434190 25801 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786267941944493-20982-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786267941944493-20982-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786267941944493-20982-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 09:32:43.453737 25801 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 09:32:43.457048 25801 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 09:32:43.457263 25801 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 09:32:43.457871 25801 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 09:32:43.458809 25801 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 09:32:43.458880 25801 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 09:32:43.458946 25801 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 09:32:43.458988 25801 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 09:32:43.469017 25801 rpc_server.cc:307] RPC server started. Bound to: 127.20.125.129:46385
I20260809 09:32:43.469049 25983 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.125.129:46385 every 8 connection(s)
I20260809 09:32:43.469951 25801 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786267941944493-20982-0/minicluster-data/ts-0/data/info.pb
I20260809 09:32:43.477420 20982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu as pid 25801
I20260809 09:32:43.477524 20982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786267941944493-20982-0/minicluster-data/ts-0/wal/instance
I20260809 09:32:43.477806 20982 external_mini_cluster.cc:1428] Setting key 1f56ff86463bd2cc481ddbd713abe13c
Aug 09 09:32:43 dist-test-slave-7kzw krb5kdc[25643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786267963, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.129@KRBTEST.COM for kudu/127.20.125.190@KRBTEST.COM
I20260809 09:32:43.485285 25768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:43.471974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.129:60481 (local address 127.20.125.190:44051)
0809 09:32:43.472176 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0809 09:32:43.472180 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 09:32:43.473057 (+ 877us) server_negotiation.cc:407] Connection header received
0809 09:32:43.474753 (+ 1696us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:32:43.474756 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:32:43.474818 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:32:43.474933 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:32:43.476234 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:43.476817 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:43.478224 (+ 1407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:43.478589 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:43.478709 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:43.481406 (+ 2697us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:32:43.481410 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:32:43.481413 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:32:43.481448 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:32:43.483056 (+ 1608us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:32:43.483627 (+ 571us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:32:43.483631 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:32:43.483633 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:32:43.483717 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:32:43.484082 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:32:43.484087 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:32:43.484089 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:32:43.484297 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:32:43.484414 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0809 09:32:43.484944 (+ 530us) server_negotiation.cc:299] Negotiation successful
0809 09:32:43.485081 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102}
I20260809 09:32:43.486121 25988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:43.472351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:44051 (local address 127.20.125.129:60481)
0809 09:32:43.472880 (+ 529us) negotiation.cc:107] Waiting for socket to connect
0809 09:32:43.472928 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0809 09:32:43.474522 (+ 1594us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:32:43.475119 (+ 597us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:32:43.475127 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:32:43.475518 (+ 391us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:32:43.476054 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:32:43.476069 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:32:43.476957 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:32:43.476961 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:32:43.478071 (+ 1110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:32:43.478082 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:32:43.478770 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:32:43.478775 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:32:43.478955 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:43.479625 (+ 670us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 09:32:43.479644 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:32:43.481219 (+ 1575us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 09:32:43.483188 (+ 1969us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:32:43.483194 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:32:43.483210 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:32:43.483497 (+ 287us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:32:43.483817 (+ 320us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:32:43.483820 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:32:43.483823 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:32:43.483977 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:32:43.484413 (+ 436us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 09:32:43.484422 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 09:32:43.484677 (+ 255us) client_negotiation.cc:769] Sending connection context
0809 09:32:43.484891 (+ 214us) client_negotiation.cc:240] Negotiation successful
0809 09:32:43.485242 (+ 351us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":292,"thread_start_us":127,"threads_started":1}
I20260809 09:32:43.487488 25985 heartbeater.cc:347] Connected to a master server at 127.20.125.190:44051
I20260809 09:32:43.487766 25985 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.20.125.130@KRBTEST.COM; defaulting to no policy
I20260809 09:32:43.488361 25985 heartbeater.cc:511] Master 127.20.125.190:44051 requested a full tablet report, sending...
I20260809 09:32:43.489971 25710 ts_manager.cc:194] Registered new tserver with Master: b702e81224404739b778e70e45663a28 (127.20.125.129:46385)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.125.130@KRBTEST.COM" created.
I20260809 09:32:43.491407 25710 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.125.129@KRBTEST.COM'} at 127.20.125.129:60481
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.125.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.130.keytab.
Entry for principal kudu/127.20.125.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.130.keytab.
I20260809 09:32:43.507325 20982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
/tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786267941944493-20982-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786267941944493-20982-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786267941944493-20982-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786267941944493-20982-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.125.130:0
--local_ip_for_outbound_sockets=127.20.125.130
--webserver_interface=127.20.125.130
--webserver_port=0
--tserver_master_addrs=127.20.125.190:44051
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.130.keytab
--principal=kudu/127.20.125.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.125.148:44507
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786267941944493-20982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786267941944493-20982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786267941944493-20982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 09:32:43.623416 25991 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 09:32:43.623749 25991 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 09:32:43.623796 25991 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 09:32:43.627844 25991 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 09:32:43.627899 25991 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 09:32:43.627981 25991 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.125.130
I20260809 09:32:43.633217 25991 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.125.148:44507
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786267941944493-20982-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786267941944493-20982-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.130.keytab
--principal=kudu/127.20.125.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.125.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786267941944493-20982-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.125.130
--webserver_port=0
--tserver_master_addrs=127.20.125.190:44051
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25991
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.125.130
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786267941944493-20982-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 08:38:55 UTC on ccycloud.quasar-njjleu.root.comops.site
build id 4575935
I20260809 09:32:43.634378 25991 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 09:32:43.635474 25991 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 09:32:43.645655 25998 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:32:43.645725 25999 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:32:43.645852 25991 server_base.cc:1034] running on GCE node
W20260809 09:32:43.645709 26006 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:32:43.646430 25991 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 09:32:43.647266 25991 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 09:32:43.648486 25991 hybrid_clock.cc:648] HybridClock initialized: now 1786267963648465 us; error 29 us; skew 500 ppm
Aug 09 09:32:43 dist-test-slave-7kzw krb5kdc[25643](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786267963, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 09:32:43.651717 25991 init.cc:377] Logged in from keytab as kudu/127.20.125.130@KRBTEST.COM (short username kudu)
I20260809 09:32:43.652602 25991 webserver.cc:460] Webserver started at http://127.20.125.130:42147/ using document root <none> and password file <none>
I20260809 09:32:43.653170 25991 fs_manager.cc:359] Metadata directory not provided
I20260809 09:32:43.653240 25991 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 09:32:43.653431 25991 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 09:32:43.655682 25991 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786267941944493-20982-0/minicluster-data/ts-1/data/instance:
uuid: "6a61a740deab4a1490056e36a0ae46ff"
format_stamp: "Formatted at 2026-08-09 09:32:43 on dist-test-slave-7kzw"
server_key: "cfecdb23971adf04a27cc0e7a18cf0f7"
server_key_iv: "24221f5a5cecae32d6e00467c6157abe"
server_key_version: "encryptionkey@0"
I20260809 09:32:43.656252 25991 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786267941944493-20982-0/minicluster-data/ts-1/wal/instance:
uuid: "6a61a740deab4a1490056e36a0ae46ff"
format_stamp: "Formatted at 2026-08-09 09:32:43 on dist-test-slave-7kzw"
server_key: "cfecdb23971adf04a27cc0e7a18cf0f7"
server_key_iv: "24221f5a5cecae32d6e00467c6157abe"
server_key_version: "encryptionkey@0"
I20260809 09:32:43.660185 25991 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 09:32:43.664526 26014 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 09:32:43.666261 25991 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260809 09:32:43.666402 25991 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786267941944493-20982-0/minicluster-data/ts-1/data,/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786267941944493-20982-0/minicluster-data/ts-1/wal
uuid: "6a61a740deab4a1490056e36a0ae46ff"
format_stamp: "Formatted at 2026-08-09 09:32:43 on dist-test-slave-7kzw"
server_key: "cfecdb23971adf04a27cc0e7a18cf0f7"
server_key_iv: "24221f5a5cecae32d6e00467c6157abe"
server_key_version: "encryptionkey@0"
I20260809 09:32:43.666592 25991 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786267941944493-20982-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786267941944493-20982-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786267941944493-20982-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 09:32:43.681865 25991 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 09:32:43.685092 25991 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 09:32:43.685259 25991 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 09:32:43.685786 25991 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 09:32:43.686690 25991 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 09:32:43.686740 25991 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 09:32:43.686784 25991 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 09:32:43.686836 25991 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 09:32:43.695508 25991 rpc_server.cc:307] RPC server started. Bound to: 127.20.125.130:36035
I20260809 09:32:43.695585 26180 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.125.130:36035 every 8 connection(s)
I20260809 09:32:43.696408 25991 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786267941944493-20982-0/minicluster-data/ts-1/data/info.pb
I20260809 09:32:43.705049 20982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu as pid 25991
I20260809 09:32:43.705204 20982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786267941944493-20982-0/minicluster-data/ts-1/wal/instance
I20260809 09:32:43.705488 20982 external_mini_cluster.cc:1428] Setting key e5c6f109bd30f52e8856eacd8ba6dadd
Aug 09 09:32:43 dist-test-slave-7kzw krb5kdc[25643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786267963, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.130@KRBTEST.COM for kudu/127.20.125.190@KRBTEST.COM
I20260809 09:32:43.710912 25768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:43.698236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.130:36623 (local address 127.20.125.190:44051)
0809 09:32:43.698377 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0809 09:32:43.698381 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 09:32:43.699161 (+ 780us) server_negotiation.cc:407] Connection header received
0809 09:32:43.700499 (+ 1338us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:32:43.700502 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:32:43.700552 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:32:43.700712 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:32:43.701888 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:43.702445 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:43.703972 (+ 1527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:43.704223 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:43.704317 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:43.707241 (+ 2924us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:32:43.707244 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:32:43.707247 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:32:43.707285 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:32:43.708767 (+ 1482us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:32:43.709310 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:32:43.709313 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:32:43.709314 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:32:43.709395 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:32:43.709746 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:32:43.709751 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:32:43.709753 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:32:43.709941 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:32:43.710063 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0809 09:32:43.710596 (+ 533us) server_negotiation.cc:299] Negotiation successful
0809 09:32:43.710740 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260809 09:32:43.711833 26183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:43.698562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:44051 (local address 127.20.125.130:36623)
0809 09:32:43.699001 (+ 439us) negotiation.cc:107] Waiting for socket to connect
0809 09:32:43.699043 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 09:32:43.700272 (+ 1229us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:32:43.700861 (+ 589us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:32:43.700868 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:32:43.701246 (+ 378us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:32:43.701728 (+ 482us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:32:43.701752 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:32:43.702715 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:32:43.702718 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:32:43.703788 (+ 1070us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:32:43.703800 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:32:43.704424 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:32:43.704428 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:32:43.704643 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:43.705384 (+ 741us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 09:32:43.705401 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:32:43.707069 (+ 1668us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 09:32:43.708891 (+ 1822us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:32:43.708897 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:32:43.708911 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:32:43.709190 (+ 279us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:32:43.709487 (+ 297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:32:43.709490 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:32:43.709495 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:32:43.709648 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:32:43.710051 (+ 403us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 09:32:43.710058 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 09:32:43.710318 (+ 260us) client_negotiation.cc:769] Sending connection context
0809 09:32:43.710540 (+ 222us) client_negotiation.cc:240] Negotiation successful
0809 09:32:43.710866 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":109,"threads_started":1}
I20260809 09:32:43.713028 26181 heartbeater.cc:347] Connected to a master server at 127.20.125.190:44051
I20260809 09:32:43.713281 26181 heartbeater.cc:464] Registering TS with master...
I20260809 09:32:43.713792 26181 heartbeater.cc:511] Master 127.20.125.190:44051 requested a full tablet report, sending...
I20260809 09:32:43.714891 25710 ts_manager.cc:194] Registered new tserver with Master: 6a61a740deab4a1490056e36a0ae46ff (127.20.125.130:36035)
I20260809 09:32:43.715803 25710 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.125.130@KRBTEST.COM'} at 127.20.125.130:36623
WARNING: no policy specified for kudu/127.20.125.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.125.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.125.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.131.keytab.
Entry for principal kudu/127.20.125.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.131.keytab.
I20260809 09:32:43.735651 20982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
/tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786267941944493-20982-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786267941944493-20982-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786267941944493-20982-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786267941944493-20982-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.125.131:0
--local_ip_for_outbound_sockets=127.20.125.131
--webserver_interface=127.20.125.131
--webserver_port=0
--tserver_master_addrs=127.20.125.190:44051
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.131.keytab
--principal=kudu/127.20.125.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.125.148:44507
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786267941944493-20982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786267941944493-20982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786267941944493-20982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 09:32:43.853029 26187 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 09:32:43.853367 26187 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 09:32:43.853443 26187 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 09:32:43.857466 26187 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 09:32:43.857559 26187 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 09:32:43.857653 26187 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.125.131
I20260809 09:32:43.863051 26187 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.125.148:44507
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786267941944493-20982-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786267941944493-20982-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.131.keytab
--principal=kudu/127.20.125.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.125.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786267941944493-20982-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.125.131
--webserver_port=0
--tserver_master_addrs=127.20.125.190:44051
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26187
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.125.131
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786267941944493-20982-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 08:38:55 UTC on ccycloud.quasar-njjleu.root.comops.site
build id 4575935
I20260809 09:32:43.864200 26187 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 09:32:43.865262 26187 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 09:32:43.875387 26203 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:32:43.875383 26194 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:32:43.875382 26196 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:32:43.875597 26187 server_base.cc:1034] running on GCE node
I20260809 09:32:43.876097 26187 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 09:32:43.876750 26187 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 09:32:43.877915 26187 hybrid_clock.cc:648] HybridClock initialized: now 1786267963877895 us; error 41 us; skew 500 ppm
Aug 09 09:32:43 dist-test-slave-7kzw krb5kdc[25643](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786267963, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 09:32:43.880837 26187 init.cc:377] Logged in from keytab as kudu/127.20.125.131@KRBTEST.COM (short username kudu)
I20260809 09:32:43.881680 26187 webserver.cc:460] Webserver started at http://127.20.125.131:42561/ using document root <none> and password file <none>
I20260809 09:32:43.882210 26187 fs_manager.cc:359] Metadata directory not provided
I20260809 09:32:43.882270 26187 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 09:32:43.882531 26187 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 09:32:43.884524 26187 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786267941944493-20982-0/minicluster-data/ts-2/data/instance:
uuid: "cae73c7952654f228c215b7ad71da1c5"
format_stamp: "Formatted at 2026-08-09 09:32:43 on dist-test-slave-7kzw"
server_key: "23f5238c720f88293487d6a384a0cf79"
server_key_iv: "710f015004ff845ea7d2d4cfa7096ad2"
server_key_version: "encryptionkey@0"
I20260809 09:32:43.885038 26187 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786267941944493-20982-0/minicluster-data/ts-2/wal/instance:
uuid: "cae73c7952654f228c215b7ad71da1c5"
format_stamp: "Formatted at 2026-08-09 09:32:43 on dist-test-slave-7kzw"
server_key: "23f5238c720f88293487d6a384a0cf79"
server_key_iv: "710f015004ff845ea7d2d4cfa7096ad2"
server_key_version: "encryptionkey@0"
I20260809 09:32:43.889549 26187 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.003s
I20260809 09:32:43.892239 26211 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 09:32:43.893560 26187 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 09:32:43.893661 26187 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786267941944493-20982-0/minicluster-data/ts-2/data,/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786267941944493-20982-0/minicluster-data/ts-2/wal
uuid: "cae73c7952654f228c215b7ad71da1c5"
format_stamp: "Formatted at 2026-08-09 09:32:43 on dist-test-slave-7kzw"
server_key: "23f5238c720f88293487d6a384a0cf79"
server_key_iv: "710f015004ff845ea7d2d4cfa7096ad2"
server_key_version: "encryptionkey@0"
I20260809 09:32:43.893751 26187 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786267941944493-20982-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786267941944493-20982-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786267941944493-20982-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 09:32:43.905985 26187 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 09:32:43.909273 26187 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 09:32:43.909432 26187 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 09:32:43.909976 26187 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 09:32:43.910794 26187 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 09:32:43.910840 26187 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 09:32:43.910881 26187 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 09:32:43.910897 26187 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 09:32:43.920444 26378 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.125.131:45513 every 8 connection(s)
I20260809 09:32:43.920411 26187 rpc_server.cc:307] RPC server started. Bound to: 127.20.125.131:45513
I20260809 09:32:43.921540 26187 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786267941944493-20982-0/minicluster-data/ts-2/data/info.pb
I20260809 09:32:43.923255 20982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu as pid 26187
I20260809 09:32:43.923388 20982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786267941944493-20982-0/minicluster-data/ts-2/wal/instance
I20260809 09:32:43.923815 20982 external_mini_cluster.cc:1428] Setting key 09df09a65825a2031eadfc89ae8ae553
Aug 09 09:32:43 dist-test-slave-7kzw krb5kdc[25643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786267963, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.131@KRBTEST.COM for kudu/127.20.125.190@KRBTEST.COM
I20260809 09:32:43.936823 25768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:43.923742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.131:49881 (local address 127.20.125.190:44051)
0809 09:32:43.923895 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0809 09:32:43.923902 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 09:32:43.925108 (+ 1206us) server_negotiation.cc:407] Connection header received
0809 09:32:43.926795 (+ 1687us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:32:43.926799 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:32:43.926860 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:32:43.926945 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:32:43.928227 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:43.928820 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:43.930082 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:43.930338 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:43.930460 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:43.933248 (+ 2788us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:32:43.933251 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:32:43.933253 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:32:43.933281 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:32:43.934785 (+ 1504us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:32:43.935300 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:32:43.935304 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:32:43.935306 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:32:43.935391 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:32:43.935745 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:32:43.935748 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:32:43.935750 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:32:43.935931 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:32:43.936037 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0809 09:32:43.936528 (+ 491us) server_negotiation.cc:299] Negotiation successful
0809 09:32:43.936655 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260809 09:32:43.937870 26385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:43.924292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:44051 (local address 127.20.125.131:49881)
0809 09:32:43.924932 (+ 640us) negotiation.cc:107] Waiting for socket to connect
0809 09:32:43.924994 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0809 09:32:43.926569 (+ 1575us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:32:43.927136 (+ 567us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:32:43.927144 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:32:43.927556 (+ 412us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:32:43.928070 (+ 514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:32:43.928083 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:32:43.928956 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:32:43.928960 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:32:43.929942 (+ 982us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:32:43.929952 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:32:43.930532 (+ 580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:32:43.930538 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:32:43.930730 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:43.931327 (+ 597us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 09:32:43.931346 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:32:43.933044 (+ 1698us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 09:32:43.934904 (+ 1860us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:32:43.934910 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:32:43.934926 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:32:43.935194 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:32:43.935483 (+ 289us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:32:43.935487 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:32:43.935489 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:32:43.935631 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:32:43.936037 (+ 406us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 09:32:43.936045 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 09:32:43.936296 (+ 251us) client_negotiation.cc:769] Sending connection context
0809 09:32:43.936487 (+ 191us) client_negotiation.cc:240] Negotiation successful
0809 09:32:43.936828 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":381,"thread_start_us":165,"threads_started":1}
I20260809 09:32:43.939368 26381 heartbeater.cc:347] Connected to a master server at 127.20.125.190:44051
I20260809 09:32:43.939579 26381 heartbeater.cc:464] Registering TS with master...
I20260809 09:32:43.940012 26381 heartbeater.cc:511] Master 127.20.125.190:44051 requested a full tablet report, sending...
I20260809 09:32:43.940990 25710 ts_manager.cc:194] Registered new tserver with Master: cae73c7952654f228c215b7ad71da1c5 (127.20.125.131:45513)
I20260809 09:32:43.941567 25710 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.125.131@KRBTEST.COM'} at 127.20.125.131:49881
I20260809 09:32:43.950325 20982 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 09 09:32:43 dist-test-slave-7kzw krb5kdc[25643](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786267963, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 09 09:32:43 dist-test-slave-7kzw krb5kdc[25643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786267963, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.20.125.190@KRBTEST.COM
I20260809 09:32:43.981344 25768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:43.970923 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34308 (local address 127.20.125.190:44051)
0809 09:32:43.971167 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0809 09:32:43.971174 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 09:32:43.971639 (+ 465us) server_negotiation.cc:407] Connection header received
0809 09:32:43.971791 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:32:43.971794 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:32:43.971860 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:32:43.971960 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:32:43.972806 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:43.973407 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:43.974363 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:43.974801 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:43.974923 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:43.977709 (+ 2786us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:32:43.977712 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:32:43.977714 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:32:43.977743 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:32:43.979371 (+ 1628us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:32:43.979863 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:32:43.979867 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:32:43.979869 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:32:43.979967 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:32:43.980274 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:32:43.980279 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:32:43.980280 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:32:43.980579 (+ 299us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:32:43.980739 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0809 09:32:43.981008 (+ 269us) server_negotiation.cc:299] Negotiation successful
0809 09:32:43.981147 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
W20260809 09:32:43.981791 25710 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:34308
I20260809 09:32:43.982719 20982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu with pid 25801
I20260809 09:32:44.008770 20982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu with pid 25991
I20260809 09:32:44.018780 20982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu with pid 26187
I20260809 09:32:44.029145 20982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu with pid 25661
2026-08-09T09:32:44Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (2430 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786267941944493-20982-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 09:32:44 dist-test-slave-7kzw krb5kdc[26402](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 09:32:44 dist-test-slave-7kzw krb5kdc[26402](info): set up 2 sockets
Aug 09 09:32:44 dist-test-slave-7kzw krb5kdc[26402](info): commencing operation
krb5kdc: starting...
W20260809 09:32:45.269860 20982 mini_kdc.cc:121] Time spent starting KDC: real 1.214s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786267941944493-20982-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786267941944493-20982-0/krb5kdc/test-user.keytab.
Aug 09 09:32:45 dist-test-slave-7kzw krb5kdc[26402](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786267965, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T09:32:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T09:32:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.125.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.125.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.125.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.190.keytab.
Entry for principal kudu/127.20.125.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.190.keytab.
I20260809 09:32:45.401993 20982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
/tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786267941944493-20982-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786267941944493-20982-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786267941944493-20982-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786267941944493-20982-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.125.190:36355
--webserver_interface=127.20.125.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.190.keytab
--principal=kudu/127.20.125.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.125.148:36949
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.125.190:36355
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786267941944493-20982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786267941944493-20982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786267941944493-20982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 09:32:45.515596 26419 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 09:32:45.515882 26419 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 09:32:45.515924 26419 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 09:32:45.520709 26419 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 09:32:45.520772 26419 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 09:32:45.520792 26419 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 09:32:45.520809 26419 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 09:32:45.520825 26419 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 09:32:45.526319 26419 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.125.148:36949
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786267941944493-20982-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786267941944493-20982-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.125.190:36355
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.190.keytab
--principal=kudu/127.20.125.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.125.190:36355
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786267941944493-20982-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.125.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26419
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786267941944493-20982-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 08:38:55 UTC on ccycloud.quasar-njjleu.root.comops.site
build id 4575935
I20260809 09:32:45.527472 26419 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 09:32:45.528537 26419 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 09:32:45.537474 26427 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:32:45.537446 26430 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:32:45.537658 26419 server_base.cc:1034] running on GCE node
W20260809 09:32:45.537894 26426 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:32:45.538565 26419 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 09:32:45.539712 26419 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 09:32:45.540891 26419 hybrid_clock.cc:648] HybridClock initialized: now 1786267965540871 us; error 33 us; skew 500 ppm
Aug 09 09:32:45 dist-test-slave-7kzw krb5kdc[26402](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786267965, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 09:32:45.543922 26419 init.cc:377] Logged in from keytab as kudu/127.20.125.190@KRBTEST.COM (short username kudu)
I20260809 09:32:45.544963 26419 webserver.cc:460] Webserver started at http://127.20.125.190:43081/ using document root <none> and password file <none>
I20260809 09:32:45.545500 26419 fs_manager.cc:359] Metadata directory not provided
I20260809 09:32:45.545563 26419 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 09:32:45.545745 26419 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 09:32:45.547894 26419 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786267941944493-20982-0/minicluster-data/master-0/data/instance:
uuid: "4092a8dc803849b7b463c581258ba5b1"
format_stamp: "Formatted at 2026-08-09 09:32:45 on dist-test-slave-7kzw"
server_key: "9551de9e568c10d0a89cc70f67708ea5"
server_key_iv: "193e0f45cfb45f7b322471ccc0888420"
server_key_version: "encryptionkey@0"
I20260809 09:32:45.548421 26419 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786267941944493-20982-0/minicluster-data/master-0/wal/instance:
uuid: "4092a8dc803849b7b463c581258ba5b1"
format_stamp: "Formatted at 2026-08-09 09:32:45 on dist-test-slave-7kzw"
server_key: "9551de9e568c10d0a89cc70f67708ea5"
server_key_iv: "193e0f45cfb45f7b322471ccc0888420"
server_key_version: "encryptionkey@0"
I20260809 09:32:45.552449 26419 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 09:32:45.555110 26439 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 09:32:45.556520 26419 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 09:32:45.556679 26419 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786267941944493-20982-0/minicluster-data/master-0/data,/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786267941944493-20982-0/minicluster-data/master-0/wal
uuid: "4092a8dc803849b7b463c581258ba5b1"
format_stamp: "Formatted at 2026-08-09 09:32:45 on dist-test-slave-7kzw"
server_key: "9551de9e568c10d0a89cc70f67708ea5"
server_key_iv: "193e0f45cfb45f7b322471ccc0888420"
server_key_version: "encryptionkey@0"
I20260809 09:32:45.556808 26419 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786267941944493-20982-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786267941944493-20982-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786267941944493-20982-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 09:32:45.573697 26419 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 09:32:45.576951 26419 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 09:32:45.577153 26419 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 09:32:45.584771 26419 rpc_server.cc:307] RPC server started. Bound to: 127.20.125.190:36355
I20260809 09:32:45.584800 26507 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.125.190:36355 every 8 connection(s)
I20260809 09:32:45.585768 26419 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786267941944493-20982-0/minicluster-data/master-0/data/info.pb
I20260809 09:32:45.588584 26508 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 09:32:45.589264 20982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu as pid 26419
I20260809 09:32:45.589402 20982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786267941944493-20982-0/minicluster-data/master-0/wal/instance
I20260809 09:32:45.589732 20982 external_mini_cluster.cc:1428] Setting key bf7bf4b47ca63afa82b6ed254d5aa48f
I20260809 09:32:45.595395 26508 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4092a8dc803849b7b463c581258ba5b1: Bootstrap starting.
I20260809 09:32:45.598037 26508 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4092a8dc803849b7b463c581258ba5b1: Neither blocks nor log segments found. Creating new log.
I20260809 09:32:45.598801 26508 log.cc:824] T 00000000000000000000000000000000 P 4092a8dc803849b7b463c581258ba5b1: Log is configured to *not* fsync() on all Append() calls
Aug 09 09:32:45 dist-test-slave-7kzw krb5kdc[26402](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786267965, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.125.190@KRBTEST.COM
I20260809 09:32:45.600827 26508 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4092a8dc803849b7b463c581258ba5b1: No bootstrap required, opened a new log
I20260809 09:32:45.604364 26508 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4092a8dc803849b7b463c581258ba5b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4092a8dc803849b7b463c581258ba5b1" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 36355 } }
I20260809 09:32:45.604583 26508 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4092a8dc803849b7b463c581258ba5b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 09:32:45.604642 26508 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4092a8dc803849b7b463c581258ba5b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4092a8dc803849b7b463c581258ba5b1, State: Initialized, Role: FOLLOWER
I20260809 09:32:45.604867 26512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:45.591162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46472 (local address 127.20.125.190:36355)
0809 09:32:45.591584 (+ 422us) server_negotiation.cc:206] Beginning negotiation
0809 09:32:45.591603 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0809 09:32:45.591649 (+ 46us) server_negotiation.cc:407] Connection header received
0809 09:32:45.592483 (+ 834us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:32:45.592506 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:32:45.592917 (+ 411us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:32:45.593277 (+ 360us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:32:45.593974 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:45.595473 (+ 1499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:45.596382 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:45.597089 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:45.597253 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:45.600130 (+ 2877us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:32:45.600135 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:32:45.600148 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:32:45.600179 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:32:45.601948 (+ 1769us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:32:45.602395 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:32:45.602401 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:32:45.602411 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:32:45.602571 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:32:45.602857 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:32:45.602861 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:32:45.602862 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:32:45.603260 (+ 398us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:32:45.603474 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0809 09:32:45.603772 (+ 298us) server_negotiation.cc:299] Negotiation successful
0809 09:32:45.604046 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":267,"thread_start_us":107,"threads_started":1}
I20260809 09:32:45.605111 26508 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4092a8dc803849b7b463c581258ba5b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4092a8dc803849b7b463c581258ba5b1" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 36355 } }
I20260809 09:32:45.605289 26508 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4092a8dc803849b7b463c581258ba5b1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 09:32:45.605361 26508 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4092a8dc803849b7b463c581258ba5b1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 09:32:45.605459 26508 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4092a8dc803849b7b463c581258ba5b1 [term 0 FOLLOWER]: Advancing to term 1
I20260809 09:32:45.606596 26508 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4092a8dc803849b7b463c581258ba5b1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4092a8dc803849b7b463c581258ba5b1" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 36355 } }
I20260809 09:32:45.606940 26508 leader_election.cc:302] T 00000000000000000000000000000000 P 4092a8dc803849b7b463c581258ba5b1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4092a8dc803849b7b463c581258ba5b1; no voters:
I20260809 09:32:45.607259 26508 leader_election.cc:288] T 00000000000000000000000000000000 P 4092a8dc803849b7b463c581258ba5b1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 09:32:45.607373 26515 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4092a8dc803849b7b463c581258ba5b1 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 09:32:45.607643 26515 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4092a8dc803849b7b463c581258ba5b1 [term 1 LEADER]: Becoming Leader. State: Replica: 4092a8dc803849b7b463c581258ba5b1, State: Running, Role: LEADER
I20260809 09:32:45.608003 26515 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4092a8dc803849b7b463c581258ba5b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4092a8dc803849b7b463c581258ba5b1" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 36355 } }
I20260809 09:32:45.608501 26508 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4092a8dc803849b7b463c581258ba5b1 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 09:32:45.609668 26518 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4092a8dc803849b7b463c581258ba5b1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4092a8dc803849b7b463c581258ba5b1. Latest consensus state: current_term: 1 leader_uuid: "4092a8dc803849b7b463c581258ba5b1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4092a8dc803849b7b463c581258ba5b1" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 36355 } } }
I20260809 09:32:45.609807 26518 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4092a8dc803849b7b463c581258ba5b1 [sys.catalog]: This master's current role is: LEADER
I20260809 09:32:45.610095 26517 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4092a8dc803849b7b463c581258ba5b1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4092a8dc803849b7b463c581258ba5b1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4092a8dc803849b7b463c581258ba5b1" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 36355 } } }
I20260809 09:32:45.610198 26517 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4092a8dc803849b7b463c581258ba5b1 [sys.catalog]: This master's current role is: LEADER
I20260809 09:32:45.610200 26536 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 09:32:45.613309 26536 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 09:32:45.618978 26536 catalog_manager.cc:1347] Generated new cluster ID: 84538e9aa63f4244bcf3d918781ad7b1
I20260809 09:32:45.619050 26536 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 09:32:45.647074 26536 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 09:32:45.648165 26536 catalog_manager.cc:1504] Loading token signing keys...
I20260809 09:32:45.657385 26536 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4092a8dc803849b7b463c581258ba5b1: Generated new TSK 0
I20260809 09:32:45.658175 26536 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.20.125.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.125.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.125.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.129.keytab.
Entry for principal kudu/127.20.125.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.129.keytab.
I20260809 09:32:45.698674 20982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
/tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786267941944493-20982-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786267941944493-20982-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786267941944493-20982-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786267941944493-20982-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.125.129:0
--local_ip_for_outbound_sockets=127.20.125.129
--webserver_interface=127.20.125.129
--webserver_port=0
--tserver_master_addrs=127.20.125.190:36355
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.129.keytab
--principal=kudu/127.20.125.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.125.148:36949
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786267941944493-20982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786267941944493-20982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786267941944493-20982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 09:32:45.813577 26552 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 09:32:45.813843 26552 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 09:32:45.813885 26552 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 09:32:45.817893 26552 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 09:32:45.817953 26552 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 09:32:45.818009 26552 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.125.129
I20260809 09:32:45.823077 26552 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.125.148:36949
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786267941944493-20982-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786267941944493-20982-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.129.keytab
--principal=kudu/127.20.125.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.125.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786267941944493-20982-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.125.129
--webserver_port=0
--tserver_master_addrs=127.20.125.190:36355
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26552
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.125.129
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786267941944493-20982-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 08:38:55 UTC on ccycloud.quasar-njjleu.root.comops.site
build id 4575935
I20260809 09:32:45.824019 26552 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 09:32:45.825143 26552 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 09:32:45.834722 26562 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:32:45.834779 26559 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:32:45.834733 26564 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:32:45.835508 26552 server_base.cc:1034] running on GCE node
I20260809 09:32:45.835901 26552 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 09:32:45.836508 26552 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 09:32:45.837679 26552 hybrid_clock.cc:648] HybridClock initialized: now 1786267965837662 us; error 28 us; skew 500 ppm
Aug 09 09:32:45 dist-test-slave-7kzw krb5kdc[26402](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786267965, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 09:32:45.840545 26552 init.cc:377] Logged in from keytab as kudu/127.20.125.129@KRBTEST.COM (short username kudu)
I20260809 09:32:45.841391 26552 webserver.cc:460] Webserver started at http://127.20.125.129:42957/ using document root <none> and password file <none>
I20260809 09:32:45.841907 26552 fs_manager.cc:359] Metadata directory not provided
I20260809 09:32:45.841967 26552 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 09:32:45.842132 26552 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 09:32:45.844190 26552 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786267941944493-20982-0/minicluster-data/ts-0/data/instance:
uuid: "4cb761c9bdd7419985bdbd39668105f3"
format_stamp: "Formatted at 2026-08-09 09:32:45 on dist-test-slave-7kzw"
server_key: "fb7a0f6a0d8b0787bcb92482fae65451"
server_key_iv: "edc4c2078210743850ea323074004396"
server_key_version: "encryptionkey@0"
I20260809 09:32:45.844705 26552 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786267941944493-20982-0/minicluster-data/ts-0/wal/instance:
uuid: "4cb761c9bdd7419985bdbd39668105f3"
format_stamp: "Formatted at 2026-08-09 09:32:45 on dist-test-slave-7kzw"
server_key: "fb7a0f6a0d8b0787bcb92482fae65451"
server_key_iv: "edc4c2078210743850ea323074004396"
server_key_version: "encryptionkey@0"
I20260809 09:32:45.848435 26552 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.003s
I20260809 09:32:45.850819 26574 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 09:32:45.852099 26552 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260809 09:32:45.852249 26552 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786267941944493-20982-0/minicluster-data/ts-0/data,/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786267941944493-20982-0/minicluster-data/ts-0/wal
uuid: "4cb761c9bdd7419985bdbd39668105f3"
format_stamp: "Formatted at 2026-08-09 09:32:45 on dist-test-slave-7kzw"
server_key: "fb7a0f6a0d8b0787bcb92482fae65451"
server_key_iv: "edc4c2078210743850ea323074004396"
server_key_version: "encryptionkey@0"
I20260809 09:32:45.852361 26552 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786267941944493-20982-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786267941944493-20982-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786267941944493-20982-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 09:32:45.876765 26552 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 09:32:45.880116 26552 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 09:32:45.880322 26552 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 09:32:45.880944 26552 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 09:32:45.881832 26552 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 09:32:45.881904 26552 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 09:32:45.881979 26552 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 09:32:45.882018 26552 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 09:32:45.893501 26552 rpc_server.cc:307] RPC server started. Bound to: 127.20.125.129:41259
I20260809 09:32:45.893524 26733 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.125.129:41259 every 8 connection(s)
I20260809 09:32:45.894403 26552 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786267941944493-20982-0/minicluster-data/ts-0/data/info.pb
I20260809 09:32:45.895706 20982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu as pid 26552
I20260809 09:32:45.895838 20982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786267941944493-20982-0/minicluster-data/ts-0/wal/instance
I20260809 09:32:45.896124 20982 external_mini_cluster.cc:1428] Setting key d150254027a12dad96930ea8d0cc7e7b
Aug 09 09:32:45 dist-test-slave-7kzw krb5kdc[26402](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786267965, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.129@KRBTEST.COM for kudu/127.20.125.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.20.125.130@KRBTEST.COM; defaulting to no policy
I20260809 09:32:45.909415 26512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:45.896360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.129:52419 (local address 127.20.125.190:36355)
0809 09:32:45.896518 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0809 09:32:45.896523 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 09:32:45.897438 (+ 915us) server_negotiation.cc:407] Connection header received
0809 09:32:45.898931 (+ 1493us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:32:45.898934 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:32:45.898987 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:32:45.899086 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:32:45.900258 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:45.900791 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:45.901961 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:45.902203 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:45.902316 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:45.905411 (+ 3095us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:32:45.905414 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:32:45.905416 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:32:45.905441 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:32:45.907137 (+ 1696us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:32:45.907684 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:32:45.907690 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:32:45.907694 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:32:45.907779 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:32:45.908207 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:32:45.908213 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:32:45.908218 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:32:45.908426 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:32:45.908511 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0809 09:32:45.909035 (+ 524us) server_negotiation.cc:299] Negotiation successful
0809 09:32:45.909218 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.125.130@KRBTEST.COM" created.
I20260809 09:32:45.910523 26738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:45.896811 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:36355 (local address 127.20.125.129:52419)
0809 09:32:45.897287 (+ 476us) negotiation.cc:107] Waiting for socket to connect
0809 09:32:45.897341 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0809 09:32:45.898722 (+ 1381us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:32:45.899259 (+ 537us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:32:45.899266 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:32:45.899646 (+ 380us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:32:45.900113 (+ 467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:32:45.900127 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:32:45.900917 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:32:45.900921 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:32:45.901834 (+ 913us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:32:45.901843 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:32:45.902357 (+ 514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:32:45.902361 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:32:45.902586 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:45.903243 (+ 657us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 09:32:45.903259 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:32:45.905229 (+ 1970us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 09:32:45.907283 (+ 2054us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:32:45.907291 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:32:45.907307 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:32:45.907575 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:32:45.907874 (+ 299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:32:45.907877 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:32:45.907879 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:32:45.908059 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:32:45.908523 (+ 464us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 09:32:45.908528 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 09:32:45.908770 (+ 242us) client_negotiation.cc:769] Sending connection context
0809 09:32:45.908930 (+ 160us) client_negotiation.cc:240] Negotiation successful
0809 09:32:45.909382 (+ 452us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":272,"thread_start_us":137,"threads_started":1}
I20260809 09:32:45.911971 26734 heartbeater.cc:347] Connected to a master server at 127.20.125.190:36355
I20260809 09:32:45.912228 26734 heartbeater.cc:464] Registering TS with master...
I20260809 09:32:45.912781 26734 heartbeater.cc:511] Master 127.20.125.190:36355 requested a full tablet report, sending...
I20260809 09:32:45.914574 26461 ts_manager.cc:194] Registered new tserver with Master: 4cb761c9bdd7419985bdbd39668105f3 (127.20.125.129:41259)
I20260809 09:32:45.916492 26461 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.125.129@KRBTEST.COM'} at 127.20.125.129:52419
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.125.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.130.keytab.
Entry for principal kudu/127.20.125.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.130.keytab.
I20260809 09:32:45.929425 20982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
/tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786267941944493-20982-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786267941944493-20982-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786267941944493-20982-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786267941944493-20982-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.125.130:0
--local_ip_for_outbound_sockets=127.20.125.130
--webserver_interface=127.20.125.130
--webserver_port=0
--tserver_master_addrs=127.20.125.190:36355
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.130.keytab
--principal=kudu/127.20.125.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.125.148:36949
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786267941944493-20982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786267941944493-20982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786267941944493-20982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 09:32:46.044836 26743 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 09:32:46.045091 26743 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 09:32:46.045135 26743 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 09:32:46.049300 26743 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 09:32:46.049358 26743 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 09:32:46.049414 26743 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.125.130
I20260809 09:32:46.054457 26743 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.125.148:36949
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786267941944493-20982-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786267941944493-20982-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.130.keytab
--principal=kudu/127.20.125.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.125.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786267941944493-20982-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.125.130
--webserver_port=0
--tserver_master_addrs=127.20.125.190:36355
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26743
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.125.130
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786267941944493-20982-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 08:38:55 UTC on ccycloud.quasar-njjleu.root.comops.site
build id 4575935
I20260809 09:32:46.055471 26743 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 09:32:46.056515 26743 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 09:32:46.066650 26752 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:32:46.066715 26743 server_base.cc:1034] running on GCE node
W20260809 09:32:46.066658 26753 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:32:46.066658 26757 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:32:46.067391 26743 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 09:32:46.068027 26743 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 09:32:46.069244 26743 hybrid_clock.cc:648] HybridClock initialized: now 1786267966069226 us; error 31 us; skew 500 ppm
Aug 09 09:32:46 dist-test-slave-7kzw krb5kdc[26402](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786267966, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 09:32:46.072433 26743 init.cc:377] Logged in from keytab as kudu/127.20.125.130@KRBTEST.COM (short username kudu)
I20260809 09:32:46.073349 26743 webserver.cc:460] Webserver started at http://127.20.125.130:46739/ using document root <none> and password file <none>
I20260809 09:32:46.073928 26743 fs_manager.cc:359] Metadata directory not provided
I20260809 09:32:46.074019 26743 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 09:32:46.074263 26743 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 09:32:46.076378 26743 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786267941944493-20982-0/minicluster-data/ts-1/data/instance:
uuid: "690e9df153be4e5ebaf81dd0d3b39f4b"
format_stamp: "Formatted at 2026-08-09 09:32:46 on dist-test-slave-7kzw"
server_key: "f2aa6c940aeb17e027a5c00b906457bd"
server_key_iv: "d00ab107c7018cc13625acfc42a5bd45"
server_key_version: "encryptionkey@0"
I20260809 09:32:46.076980 26743 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786267941944493-20982-0/minicluster-data/ts-1/wal/instance:
uuid: "690e9df153be4e5ebaf81dd0d3b39f4b"
format_stamp: "Formatted at 2026-08-09 09:32:46 on dist-test-slave-7kzw"
server_key: "f2aa6c940aeb17e027a5c00b906457bd"
server_key_iv: "d00ab107c7018cc13625acfc42a5bd45"
server_key_version: "encryptionkey@0"
I20260809 09:32:46.080956 26743 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 09:32:46.083360 26769 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 09:32:46.084491 26743 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 09:32:46.084635 26743 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786267941944493-20982-0/minicluster-data/ts-1/data,/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786267941944493-20982-0/minicluster-data/ts-1/wal
uuid: "690e9df153be4e5ebaf81dd0d3b39f4b"
format_stamp: "Formatted at 2026-08-09 09:32:46 on dist-test-slave-7kzw"
server_key: "f2aa6c940aeb17e027a5c00b906457bd"
server_key_iv: "d00ab107c7018cc13625acfc42a5bd45"
server_key_version: "encryptionkey@0"
I20260809 09:32:46.084776 26743 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786267941944493-20982-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786267941944493-20982-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786267941944493-20982-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 09:32:46.103004 26743 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 09:32:46.105964 26743 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 09:32:46.106176 26743 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 09:32:46.106792 26743 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 09:32:46.107622 26743 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 09:32:46.107697 26743 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 09:32:46.107769 26743 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 09:32:46.107815 26743 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 09:32:46.117098 26743 rpc_server.cc:307] RPC server started. Bound to: 127.20.125.130:41843
I20260809 09:32:46.117157 26929 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.125.130:41843 every 8 connection(s)
I20260809 09:32:46.118013 26743 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786267941944493-20982-0/minicluster-data/ts-1/data/info.pb
I20260809 09:32:46.127105 20982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu as pid 26743
I20260809 09:32:46.127247 20982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786267941944493-20982-0/minicluster-data/ts-1/wal/instance
I20260809 09:32:46.127609 20982 external_mini_cluster.cc:1428] Setting key d88046be20c13dca0d8fea21ba4e7d97
Aug 09 09:32:46 dist-test-slave-7kzw krb5kdc[26402](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786267966, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.130@KRBTEST.COM for kudu/127.20.125.190@KRBTEST.COM
I20260809 09:32:46.132998 26512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:46.119865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.130:60785 (local address 127.20.125.190:36355)
0809 09:32:46.120033 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0809 09:32:46.120038 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 09:32:46.120888 (+ 850us) server_negotiation.cc:407] Connection header received
0809 09:32:46.122417 (+ 1529us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:32:46.122420 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:32:46.122535 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:32:46.122619 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:32:46.124048 (+ 1429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:46.124575 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:46.125822 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:46.126052 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:46.126145 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:46.129068 (+ 2923us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:32:46.129071 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:32:46.129073 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:32:46.129120 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:32:46.130833 (+ 1713us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:32:46.131335 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:32:46.131340 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:32:46.131344 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:32:46.131449 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:32:46.131872 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:32:46.131875 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:32:46.131876 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:32:46.132047 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:32:46.132133 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0809 09:32:46.132767 (+ 634us) server_negotiation.cc:299] Negotiation successful
0809 09:32:46.132877 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260809 09:32:46.134159 26933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:46.120227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:36355 (local address 127.20.125.130:60785)
0809 09:32:46.120728 (+ 501us) negotiation.cc:107] Waiting for socket to connect
0809 09:32:46.120777 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0809 09:32:46.122186 (+ 1409us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:32:46.122952 (+ 766us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:32:46.122959 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:32:46.123373 (+ 414us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:32:46.123892 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:32:46.123907 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:32:46.124708 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:32:46.124710 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:32:46.125685 (+ 975us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:32:46.125695 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:32:46.126205 (+ 510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:32:46.126208 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:32:46.126355 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:46.127046 (+ 691us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 09:32:46.127068 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:32:46.128876 (+ 1808us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 09:32:46.130947 (+ 2071us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:32:46.130952 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:32:46.130963 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:32:46.131236 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:32:46.131563 (+ 327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:32:46.131568 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:32:46.131571 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:32:46.131762 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:32:46.132157 (+ 395us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 09:32:46.132166 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 09:32:46.132498 (+ 332us) client_negotiation.cc:769] Sending connection context
0809 09:32:46.132718 (+ 220us) client_negotiation.cc:240] Negotiation successful
0809 09:32:46.133077 (+ 359us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":287,"thread_start_us":116,"threads_started":1}
I20260809 09:32:46.135725 26930 heartbeater.cc:347] Connected to a master server at 127.20.125.190:36355
I20260809 09:32:46.135982 26930 heartbeater.cc:464] Registering TS with master...
I20260809 09:32:46.136478 26930 heartbeater.cc:511] Master 127.20.125.190:36355 requested a full tablet report, sending...
I20260809 09:32:46.137514 26461 ts_manager.cc:194] Registered new tserver with Master: 690e9df153be4e5ebaf81dd0d3b39f4b (127.20.125.130:41843)
I20260809 09:32:46.138083 26461 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.125.130@KRBTEST.COM'} at 127.20.125.130:60785
WARNING: no policy specified for kudu/127.20.125.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.125.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.125.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.131.keytab.
Entry for principal kudu/127.20.125.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.131.keytab.
I20260809 09:32:46.156981 20982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
/tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786267941944493-20982-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786267941944493-20982-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786267941944493-20982-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786267941944493-20982-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.125.131:0
--local_ip_for_outbound_sockets=127.20.125.131
--webserver_interface=127.20.125.131
--webserver_port=0
--tserver_master_addrs=127.20.125.190:36355
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.131.keytab
--principal=kudu/127.20.125.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.125.148:36949
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786267941944493-20982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786267941944493-20982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786267941944493-20982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 09:32:46.271633 26937 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 09:32:46.271898 26937 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 09:32:46.271940 26937 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 09:32:46.276086 26937 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 09:32:46.276152 26937 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 09:32:46.276212 26937 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.125.131
I20260809 09:32:46.281610 26937 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.125.148:36949
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786267941944493-20982-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786267941944493-20982-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.131.keytab
--principal=kudu/127.20.125.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.125.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786267941944493-20982-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.125.131
--webserver_port=0
--tserver_master_addrs=127.20.125.190:36355
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26937
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.125.131
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786267941944493-20982-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 08:38:55 UTC on ccycloud.quasar-njjleu.root.comops.site
build id 4575935
I20260809 09:32:46.282739 26937 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 09:32:46.283749 26937 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 09:32:46.293470 26943 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:32:46.293532 26945 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:32:46.293501 26948 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:32:46.293859 26937 server_base.cc:1034] running on GCE node
I20260809 09:32:46.294269 26937 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 09:32:46.294955 26937 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 09:32:46.296115 26937 hybrid_clock.cc:648] HybridClock initialized: now 1786267966296081 us; error 46 us; skew 500 ppm
Aug 09 09:32:46 dist-test-slave-7kzw krb5kdc[26402](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786267966, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 09:32:46.298842 26937 init.cc:377] Logged in from keytab as kudu/127.20.125.131@KRBTEST.COM (short username kudu)
I20260809 09:32:46.299742 26937 webserver.cc:460] Webserver started at http://127.20.125.131:41337/ using document root <none> and password file <none>
I20260809 09:32:46.300323 26937 fs_manager.cc:359] Metadata directory not provided
I20260809 09:32:46.300414 26937 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 09:32:46.300642 26937 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 09:32:46.302757 26937 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786267941944493-20982-0/minicluster-data/ts-2/data/instance:
uuid: "3a979c9cba5640f68c80d9c4c48235b6"
format_stamp: "Formatted at 2026-08-09 09:32:46 on dist-test-slave-7kzw"
server_key: "9a1a0c2dd02182c36472a787945413ba"
server_key_iv: "8d75aefd14edc87163e8e2bd92264052"
server_key_version: "encryptionkey@0"
I20260809 09:32:46.303342 26937 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786267941944493-20982-0/minicluster-data/ts-2/wal/instance:
uuid: "3a979c9cba5640f68c80d9c4c48235b6"
format_stamp: "Formatted at 2026-08-09 09:32:46 on dist-test-slave-7kzw"
server_key: "9a1a0c2dd02182c36472a787945413ba"
server_key_iv: "8d75aefd14edc87163e8e2bd92264052"
server_key_version: "encryptionkey@0"
I20260809 09:32:46.307037 26937 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260809 09:32:46.309608 26955 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 09:32:46.310928 26937 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 09:32:46.311059 26937 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786267941944493-20982-0/minicluster-data/ts-2/data,/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786267941944493-20982-0/minicluster-data/ts-2/wal
uuid: "3a979c9cba5640f68c80d9c4c48235b6"
format_stamp: "Formatted at 2026-08-09 09:32:46 on dist-test-slave-7kzw"
server_key: "9a1a0c2dd02182c36472a787945413ba"
server_key_iv: "8d75aefd14edc87163e8e2bd92264052"
server_key_version: "encryptionkey@0"
I20260809 09:32:46.311165 26937 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786267941944493-20982-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786267941944493-20982-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786267941944493-20982-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 09:32:46.322230 26937 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 09:32:46.325042 26937 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 09:32:46.325254 26937 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 09:32:46.325821 26937 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 09:32:46.326681 26937 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 09:32:46.326756 26937 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 09:32:46.326828 26937 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 09:32:46.326865 26937 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 09:32:46.335994 26937 rpc_server.cc:307] RPC server started. Bound to: 127.20.125.131:39039
I20260809 09:32:46.336031 27120 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.125.131:39039 every 8 connection(s)
I20260809 09:32:46.336884 26937 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786267941944493-20982-0/minicluster-data/ts-2/data/info.pb
I20260809 09:32:46.343632 20982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu as pid 26937
I20260809 09:32:46.343784 20982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786267941944493-20982-0/minicluster-data/ts-2/wal/instance
I20260809 09:32:46.344085 20982 external_mini_cluster.cc:1428] Setting key b0302607fa0ba8e94e588dadbe7e3990
Aug 09 09:32:46 dist-test-slave-7kzw krb5kdc[26402](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786267966, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.131@KRBTEST.COM for kudu/127.20.125.190@KRBTEST.COM
I20260809 09:32:46.351751 26512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:46.338645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.131:59717 (local address 127.20.125.190:36355)
0809 09:32:46.338810 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0809 09:32:46.338815 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 09:32:46.339659 (+ 844us) server_negotiation.cc:407] Connection header received
0809 09:32:46.341009 (+ 1350us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:32:46.341012 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:32:46.341062 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:32:46.341138 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:32:46.342267 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:46.342897 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:46.344497 (+ 1600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:46.344725 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:46.344847 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:46.347722 (+ 2875us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:32:46.347726 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:32:46.347729 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:32:46.347757 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:32:46.349531 (+ 1774us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:32:46.350037 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:32:46.350039 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:32:46.350040 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:32:46.350124 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:32:46.350601 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:32:46.350604 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:32:46.350605 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:32:46.350851 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:32:46.350942 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0809 09:32:46.351463 (+ 521us) server_negotiation.cc:299] Negotiation successful
0809 09:32:46.351576 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260809 09:32:46.352547 27126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:46.339045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:36355 (local address 127.20.125.131:59717)
0809 09:32:46.339506 (+ 461us) negotiation.cc:107] Waiting for socket to connect
0809 09:32:46.339548 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 09:32:46.340800 (+ 1252us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:32:46.341312 (+ 512us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:32:46.341318 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:32:46.341666 (+ 348us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:32:46.342129 (+ 463us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:32:46.342142 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:32:46.343024 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:32:46.343027 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:32:46.344330 (+ 1303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:32:46.344342 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:32:46.344888 (+ 546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:32:46.344892 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:32:46.345131 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:46.345773 (+ 642us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 09:32:46.345790 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:32:46.347488 (+ 1698us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 09:32:46.349654 (+ 2166us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:32:46.349659 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:32:46.349673 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:32:46.349944 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:32:46.350254 (+ 310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:32:46.350257 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:32:46.350258 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:32:46.350436 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:32:46.350958 (+ 522us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 09:32:46.350963 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 09:32:46.351220 (+ 257us) client_negotiation.cc:769] Sending connection context
0809 09:32:46.351380 (+ 160us) client_negotiation.cc:240] Negotiation successful
0809 09:32:46.351689 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":281,"thread_start_us":131,"threads_started":1}
I20260809 09:32:46.353790 27123 heartbeater.cc:347] Connected to a master server at 127.20.125.190:36355
I20260809 09:32:46.354049 27123 heartbeater.cc:464] Registering TS with master...
I20260809 09:32:46.354611 27123 heartbeater.cc:511] Master 127.20.125.190:36355 requested a full tablet report, sending...
I20260809 09:32:46.355686 26461 ts_manager.cc:194] Registered new tserver with Master: 3a979c9cba5640f68c80d9c4c48235b6 (127.20.125.131:39039)
I20260809 09:32:46.356462 26461 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.125.131@KRBTEST.COM'} at 127.20.125.131:59717
I20260809 09:32:46.358234 20982 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 09 09:32:46 dist-test-slave-7kzw krb5kdc[26402](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786267966, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 09:32:46 dist-test-slave-7kzw krb5kdc[26402](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786267966, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.125.129@KRBTEST.COM
I20260809 09:32:46.387162 27131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:46.377289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56534 (local address 127.20.125.129:41259)
0809 09:32:46.377594 (+ 305us) server_negotiation.cc:206] Beginning negotiation
0809 09:32:46.377609 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 09:32:46.377634 (+ 25us) server_negotiation.cc:407] Connection header received
0809 09:32:46.377688 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:32:46.377693 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:32:46.377891 (+ 198us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:32:46.378090 (+ 199us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:32:46.378834 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:46.379649 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:46.380662 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:46.381032 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:46.381132 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:46.383612 (+ 2480us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:32:46.383629 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:32:46.383646 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:32:46.383670 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:32:46.385362 (+ 1692us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:32:46.385819 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:32:46.385824 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:32:46.385825 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:32:46.385907 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:32:46.386149 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:32:46.386151 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:32:46.386152 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:32:46.386400 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:32:46.386634 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0809 09:32:46.386876 (+ 242us) server_negotiation.cc:299] Negotiation successful
0809 09:32:46.387004 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":199,"thread_start_us":103,"threads_started":1}
Aug 09 09:32:46 dist-test-slave-7kzw krb5kdc[26402](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786267966, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.125.190@KRBTEST.COM
I20260809 09:32:46.398443 26512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:46.390277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46480 (local address 127.20.125.190:36355)
0809 09:32:46.390417 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0809 09:32:46.390423 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 09:32:46.390506 (+ 83us) server_negotiation.cc:407] Connection header received
0809 09:32:46.390764 (+ 258us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:32:46.390767 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:32:46.390819 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:32:46.390903 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:32:46.391494 (+ 591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:46.392013 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:46.392783 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:46.393011 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:46.393114 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:46.395416 (+ 2302us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:32:46.395420 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:32:46.395427 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:32:46.395451 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:32:46.396769 (+ 1318us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:32:46.397179 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:32:46.397188 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:32:46.397189 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:32:46.397266 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:32:46.397530 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:32:46.397534 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:32:46.397536 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:32:46.397763 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:32:46.397906 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0809 09:32:46.398135 (+ 229us) server_negotiation.cc:299] Negotiation successful
0809 09:32:46.398279 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
W20260809 09:32:46.399037 26461 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46480
I20260809 09:32:46.399720 20982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu with pid 26552
I20260809 09:32:46.416359 20982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu with pid 26743
I20260809 09:32:46.426638 20982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu with pid 26937
I20260809 09:32:46.440811 20982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu with pid 26419
2026-08-09T09:32:46Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (2413 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786267941944493-20982-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 09:32:46 dist-test-slave-7kzw krb5kdc[27137](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 09:32:46 dist-test-slave-7kzw krb5kdc[27137](info): set up 2 sockets
Aug 09 09:32:46 dist-test-slave-7kzw krb5kdc[27137](info): commencing operation
krb5kdc: starting...
W20260809 09:32:47.761144 20982 mini_kdc.cc:121] Time spent starting KDC: real 1.292s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786267941944493-20982-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786267941944493-20982-0/krb5kdc/test-user.keytab.
Aug 09 09:32:47 dist-test-slave-7kzw krb5kdc[27137](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786267967, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T09:32:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T09:32:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.125.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.125.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.125.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.190.keytab.
Entry for principal kudu/127.20.125.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.190.keytab.
I20260809 09:32:47.893318 20982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
/tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786267941944493-20982-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786267941944493-20982-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786267941944493-20982-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786267941944493-20982-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.125.190:39399
--webserver_interface=127.20.125.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.190.keytab
--principal=kudu/127.20.125.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.125.148:40853
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.125.190:39399
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786267941944493-20982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786267941944493-20982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786267941944493-20982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 09:32:48.009696 27158 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 09:32:48.010010 27158 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 09:32:48.010054 27158 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 09:32:48.014115 27158 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 09:32:48.014180 27158 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 09:32:48.014200 27158 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 09:32:48.014217 27158 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 09:32:48.014232 27158 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 09:32:48.019840 27158 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.125.148:40853
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786267941944493-20982-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786267941944493-20982-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.125.190:39399
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.190.keytab
--principal=kudu/127.20.125.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.125.190:39399
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786267941944493-20982-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.125.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27158
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786267941944493-20982-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 08:38:55 UTC on ccycloud.quasar-njjleu.root.comops.site
build id 4575935
I20260809 09:32:48.020882 27158 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 09:32:48.021891 27158 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 09:32:48.030499 27170 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:32:48.030663 27158 server_base.cc:1034] running on GCE node
W20260809 09:32:48.030691 27165 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:32:48.030890 27168 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:32:48.031519 27158 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 09:32:48.032637 27158 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 09:32:48.033847 27158 hybrid_clock.cc:648] HybridClock initialized: now 1786267968033813 us; error 49 us; skew 500 ppm
Aug 09 09:32:48 dist-test-slave-7kzw krb5kdc[27137](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786267968, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 09:32:48.037041 27158 init.cc:377] Logged in from keytab as kudu/127.20.125.190@KRBTEST.COM (short username kudu)
I20260809 09:32:48.037901 27158 fs_manager.cc:359] Metadata directory not provided
I20260809 09:32:48.038000 27158 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 09:32:48.038245 27158 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 09:32:48.040374 27158 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786267941944493-20982-0/minicluster-data/master-0/data/instance:
uuid: "715e779361714d09a66e9084cc8b8767"
format_stamp: "Formatted at 2026-08-09 09:32:48 on dist-test-slave-7kzw"
server_key: "6f372f7dde1087befb7d369b0985c8b9"
server_key_iv: "50057ad12535a8a5da5e40da06f0987e"
server_key_version: "encryptionkey@0"
I20260809 09:32:48.040949 27158 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786267941944493-20982-0/minicluster-data/master-0/wal/instance:
uuid: "715e779361714d09a66e9084cc8b8767"
format_stamp: "Formatted at 2026-08-09 09:32:48 on dist-test-slave-7kzw"
server_key: "6f372f7dde1087befb7d369b0985c8b9"
server_key_iv: "50057ad12535a8a5da5e40da06f0987e"
server_key_version: "encryptionkey@0"
I20260809 09:32:48.044756 27158 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 09:32:48.047286 27179 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 09:32:48.048760 27158 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 09:32:48.048916 27158 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786267941944493-20982-0/minicluster-data/master-0/data,/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786267941944493-20982-0/minicluster-data/master-0/wal
uuid: "715e779361714d09a66e9084cc8b8767"
format_stamp: "Formatted at 2026-08-09 09:32:48 on dist-test-slave-7kzw"
server_key: "6f372f7dde1087befb7d369b0985c8b9"
server_key_iv: "50057ad12535a8a5da5e40da06f0987e"
server_key_version: "encryptionkey@0"
I20260809 09:32:48.049033 27158 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786267941944493-20982-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786267941944493-20982-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786267941944493-20982-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 09:32:48.063565 27158 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 09:32:48.066916 27158 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 09:32:48.067121 27158 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 09:32:48.074432 27158 rpc_server.cc:307] RPC server started. Bound to: 127.20.125.190:39399
I20260809 09:32:48.074791 27257 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.125.190:39399 every 8 connection(s)
I20260809 09:32:48.075199 27158 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786267941944493-20982-0/minicluster-data/master-0/data/info.pb
I20260809 09:32:48.078073 27258 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 09:32:48.082255 20982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu as pid 27158
I20260809 09:32:48.082419 20982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786267941944493-20982-0/minicluster-data/master-0/wal/instance
I20260809 09:32:48.082741 20982 external_mini_cluster.cc:1428] Setting key 451d0557f43aad94d1571cb123afe293
I20260809 09:32:48.084419 27258 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 715e779361714d09a66e9084cc8b8767: Bootstrap starting.
I20260809 09:32:48.087059 27258 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 715e779361714d09a66e9084cc8b8767: Neither blocks nor log segments found. Creating new log.
I20260809 09:32:48.087911 27258 log.cc:824] T 00000000000000000000000000000000 P 715e779361714d09a66e9084cc8b8767: Log is configured to *not* fsync() on all Append() calls
I20260809 09:32:48.090238 27258 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 715e779361714d09a66e9084cc8b8767: No bootstrap required, opened a new log
Aug 09 09:32:48 dist-test-slave-7kzw krb5kdc[27137](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786267967, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.125.190@KRBTEST.COM
I20260809 09:32:48.094735 27258 raft_consensus.cc:348] T 00000000000000000000000000000000 P 715e779361714d09a66e9084cc8b8767 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "715e779361714d09a66e9084cc8b8767" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 39399 } }
I20260809 09:32:48.094995 27258 raft_consensus.cc:374] T 00000000000000000000000000000000 P 715e779361714d09a66e9084cc8b8767 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 09:32:48.095080 27258 raft_consensus.cc:729] T 00000000000000000000000000000000 P 715e779361714d09a66e9084cc8b8767 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 715e779361714d09a66e9084cc8b8767, State: Initialized, Role: FOLLOWER
I20260809 09:32:48.095579 27258 consensus_queue.cc:260] T 00000000000000000000000000000000 P 715e779361714d09a66e9084cc8b8767 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "715e779361714d09a66e9084cc8b8767" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 39399 } }
I20260809 09:32:48.095750 27258 raft_consensus.cc:388] T 00000000000000000000000000000000 P 715e779361714d09a66e9084cc8b8767 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 09:32:48.095841 27258 raft_consensus.cc:482] T 00000000000000000000000000000000 P 715e779361714d09a66e9084cc8b8767 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 09:32:48.095973 27258 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 715e779361714d09a66e9084cc8b8767 [term 0 FOLLOWER]: Advancing to term 1
I20260809 09:32:48.096925 27258 raft_consensus.cc:504] T 00000000000000000000000000000000 P 715e779361714d09a66e9084cc8b8767 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "715e779361714d09a66e9084cc8b8767" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 39399 } }
I20260809 09:32:48.097301 27258 leader_election.cc:302] T 00000000000000000000000000000000 P 715e779361714d09a66e9084cc8b8767 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 715e779361714d09a66e9084cc8b8767; no voters:
I20260809 09:32:48.097723 27258 leader_election.cc:288] T 00000000000000000000000000000000 P 715e779361714d09a66e9084cc8b8767 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 09:32:48.097805 27264 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 715e779361714d09a66e9084cc8b8767 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 09:32:48.098019 27264 raft_consensus.cc:686] T 00000000000000000000000000000000 P 715e779361714d09a66e9084cc8b8767 [term 1 LEADER]: Becoming Leader. State: Replica: 715e779361714d09a66e9084cc8b8767, State: Running, Role: LEADER
I20260809 09:32:48.098424 27264 consensus_queue.cc:237] T 00000000000000000000000000000000 P 715e779361714d09a66e9084cc8b8767 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "715e779361714d09a66e9084cc8b8767" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 39399 } }
I20260809 09:32:48.098945 27262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:48.084263 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42656 (local address 127.20.125.190:39399)
0809 09:32:48.084711 (+ 448us) server_negotiation.cc:206] Beginning negotiation
0809 09:32:48.084734 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0809 09:32:48.084773 (+ 39us) server_negotiation.cc:407] Connection header received
0809 09:32:48.085449 (+ 676us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:32:48.085467 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:32:48.085797 (+ 330us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:32:48.086103 (+ 306us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:32:48.087178 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:48.088383 (+ 1205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:48.089687 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:48.090218 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:48.090414 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:48.093440 (+ 3026us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:32:48.093446 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:32:48.093459 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:32:48.093488 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:32:48.095499 (+ 2011us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:32:48.096056 (+ 557us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:32:48.096078 (+ 22us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:32:48.096087 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:32:48.096220 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:32:48.096524 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:32:48.096529 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:32:48.096531 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:32:48.096948 (+ 417us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:32:48.097175 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0809 09:32:48.097495 (+ 320us) server_negotiation.cc:299] Negotiation successful
0809 09:32:48.097857 (+ 362us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":298,"thread_start_us":146,"threads_started":1}
I20260809 09:32:48.099114 27258 sys_catalog.cc:564] T 00000000000000000000000000000000 P 715e779361714d09a66e9084cc8b8767 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 09:32:48.100476 27265 sys_catalog.cc:455] T 00000000000000000000000000000000 P 715e779361714d09a66e9084cc8b8767 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "715e779361714d09a66e9084cc8b8767" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "715e779361714d09a66e9084cc8b8767" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 39399 } } }
I20260809 09:32:48.100612 27265 sys_catalog.cc:458] T 00000000000000000000000000000000 P 715e779361714d09a66e9084cc8b8767 [sys.catalog]: This master's current role is: LEADER
I20260809 09:32:48.101032 27266 sys_catalog.cc:455] T 00000000000000000000000000000000 P 715e779361714d09a66e9084cc8b8767 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 715e779361714d09a66e9084cc8b8767. Latest consensus state: current_term: 1 leader_uuid: "715e779361714d09a66e9084cc8b8767" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "715e779361714d09a66e9084cc8b8767" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 39399 } } }
I20260809 09:32:48.101187 27266 sys_catalog.cc:458] T 00000000000000000000000000000000 P 715e779361714d09a66e9084cc8b8767 [sys.catalog]: This master's current role is: LEADER
I20260809 09:32:48.101687 27277 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 09:32:48.104980 27277 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 09:32:48.111442 27277 catalog_manager.cc:1347] Generated new cluster ID: 43c2780614064eef88c3f04cb86ba567
I20260809 09:32:48.111521 27277 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 09:32:48.119982 27277 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 09:32:48.121356 27277 catalog_manager.cc:1504] Loading token signing keys...
I20260809 09:32:48.131871 27277 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 715e779361714d09a66e9084cc8b8767: Generated new TSK 0
I20260809 09:32:48.132586 27277 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.20.125.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.125.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.125.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.129.keytab.
Entry for principal kudu/127.20.125.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.129.keytab.
I20260809 09:32:48.173758 20982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
/tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786267941944493-20982-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786267941944493-20982-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786267941944493-20982-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786267941944493-20982-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.125.129:0
--local_ip_for_outbound_sockets=127.20.125.129
--webserver_interface=127.20.125.129
--webserver_port=0
--tserver_master_addrs=127.20.125.190:39399
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.129.keytab
--principal=kudu/127.20.125.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.125.148:40853
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786267941944493-20982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786267941944493-20982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786267941944493-20982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 09:32:48.310016 27298 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 09:32:48.310289 27298 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 09:32:48.310338 27298 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 09:32:48.314349 27298 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 09:32:48.314419 27298 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 09:32:48.314496 27298 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.125.129
I20260809 09:32:48.319813 27298 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.125.148:40853
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786267941944493-20982-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786267941944493-20982-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.129.keytab
--principal=kudu/127.20.125.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.125.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786267941944493-20982-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.125.129
--webserver_port=0
--tserver_master_addrs=127.20.125.190:39399
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27298
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.125.129
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786267941944493-20982-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 08:38:55 UTC on ccycloud.quasar-njjleu.root.comops.site
build id 4575935
I20260809 09:32:48.320817 27298 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 09:32:48.321785 27298 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 09:32:48.331013 27310 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:32:48.331060 27306 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:32:48.331269 27298 server_base.cc:1034] running on GCE node
W20260809 09:32:48.331013 27308 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:32:48.331969 27298 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 09:32:48.332683 27298 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 09:32:48.333865 27298 hybrid_clock.cc:648] HybridClock initialized: now 1786267968333849 us; error 25 us; skew 500 ppm
Aug 09 09:32:48 dist-test-slave-7kzw krb5kdc[27137](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786267968, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 09:32:48.336931 27298 init.cc:377] Logged in from keytab as kudu/127.20.125.129@KRBTEST.COM (short username kudu)
I20260809 09:32:48.337770 27298 fs_manager.cc:359] Metadata directory not provided
I20260809 09:32:48.337873 27298 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 09:32:48.338136 27298 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 09:32:48.340348 27298 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786267941944493-20982-0/minicluster-data/ts-0/data/instance:
uuid: "6f03935c0ae54f6c8c48865f4b870ef8"
format_stamp: "Formatted at 2026-08-09 09:32:48 on dist-test-slave-7kzw"
server_key: "cb056fd10728a8a913d6a22554045315"
server_key_iv: "b7fe6b4f63f32b0a2618b52e091161ad"
server_key_version: "encryptionkey@0"
I20260809 09:32:48.340941 27298 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786267941944493-20982-0/minicluster-data/ts-0/wal/instance:
uuid: "6f03935c0ae54f6c8c48865f4b870ef8"
format_stamp: "Formatted at 2026-08-09 09:32:48 on dist-test-slave-7kzw"
server_key: "cb056fd10728a8a913d6a22554045315"
server_key_iv: "b7fe6b4f63f32b0a2618b52e091161ad"
server_key_version: "encryptionkey@0"
I20260809 09:32:48.344735 27298 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260809 09:32:48.347225 27317 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 09:32:48.348435 27298 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 09:32:48.348703 27298 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786267941944493-20982-0/minicluster-data/ts-0/data,/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786267941944493-20982-0/minicluster-data/ts-0/wal
uuid: "6f03935c0ae54f6c8c48865f4b870ef8"
format_stamp: "Formatted at 2026-08-09 09:32:48 on dist-test-slave-7kzw"
server_key: "cb056fd10728a8a913d6a22554045315"
server_key_iv: "b7fe6b4f63f32b0a2618b52e091161ad"
server_key_version: "encryptionkey@0"
I20260809 09:32:48.348816 27298 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786267941944493-20982-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786267941944493-20982-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786267941944493-20982-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 09:32:48.369520 27298 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 09:32:48.372823 27298 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 09:32:48.373044 27298 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 09:32:48.373656 27298 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 09:32:48.374565 27298 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 09:32:48.374636 27298 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 09:32:48.374712 27298 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 09:32:48.374752 27298 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 09:32:48.384052 27298 rpc_server.cc:307] RPC server started. Bound to: 127.20.125.129:41219
I20260809 09:32:48.384078 27497 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.125.129:41219 every 8 connection(s)
I20260809 09:32:48.384769 27298 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786267941944493-20982-0/minicluster-data/ts-0/data/info.pb
I20260809 09:32:48.393406 20982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu as pid 27298
I20260809 09:32:48.393555 20982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786267941944493-20982-0/minicluster-data/ts-0/wal/instance
I20260809 09:32:48.393911 20982 external_mini_cluster.cc:1428] Setting key e12f45fb2d02828339fc880f7e2e793f
Aug 09 09:32:48 dist-test-slave-7kzw krb5kdc[27137](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786267968, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.129@KRBTEST.COM for kudu/127.20.125.190@KRBTEST.COM
I20260809 09:32:48.399557 27262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:48.386743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.129:43937 (local address 127.20.125.190:39399)
0809 09:32:48.386910 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0809 09:32:48.386915 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 09:32:48.388163 (+ 1248us) server_negotiation.cc:407] Connection header received
0809 09:32:48.389008 (+ 845us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:32:48.389011 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:32:48.389064 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:32:48.389151 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:32:48.390409 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:48.391078 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:48.392229 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:48.392506 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:48.392610 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:48.395653 (+ 3043us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:32:48.395661 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:32:48.395664 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:32:48.395686 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:32:48.397290 (+ 1604us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:32:48.397875 (+ 585us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:32:48.397878 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:32:48.397880 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:32:48.397958 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:32:48.398317 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:32:48.398322 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:32:48.398324 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:32:48.398581 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:32:48.398741 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0809 09:32:48.399261 (+ 520us) server_negotiation.cc:299] Negotiation successful
0809 09:32:48.399394 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260809 09:32:48.400549 27501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:48.387121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:39399 (local address 127.20.125.129:43937)
0809 09:32:48.387984 (+ 863us) negotiation.cc:107] Waiting for socket to connect
0809 09:32:48.388036 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0809 09:32:48.388802 (+ 766us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:32:48.389340 (+ 538us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:32:48.389346 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:32:48.389718 (+ 372us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:32:48.390255 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:32:48.390271 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:32:48.391206 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:32:48.391209 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:32:48.392105 (+ 896us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:32:48.392115 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:32:48.392709 (+ 594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:32:48.392713 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:32:48.392873 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:48.393664 (+ 791us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 09:32:48.393687 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:32:48.395489 (+ 1802us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 09:32:48.397431 (+ 1942us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:32:48.397439 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:32:48.397455 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:32:48.397753 (+ 298us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:32:48.398052 (+ 299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:32:48.398056 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:32:48.398058 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:32:48.398219 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:32:48.398738 (+ 519us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 09:32:48.398746 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 09:32:48.399010 (+ 264us) client_negotiation.cc:769] Sending connection context
0809 09:32:48.399219 (+ 209us) client_negotiation.cc:240] Negotiation successful
0809 09:32:48.399612 (+ 393us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":293,"thread_start_us":122,"threads_started":1}
I20260809 09:32:48.401979 27499 heartbeater.cc:347] Connected to a master server at 127.20.125.190:39399
I20260809 09:32:48.402263 27499 heartbeater.cc:464] Registering TS with master...
I20260809 09:32:48.402839 27499 heartbeater.cc:511] Master 127.20.125.190:39399 requested a full tablet report, sending...
I20260809 09:32:48.404641 27200 ts_manager.cc:194] Registered new tserver with Master: 6f03935c0ae54f6c8c48865f4b870ef8 (127.20.125.129:41219)
WARNING: no policy specified for kudu/127.20.125.130@KRBTEST.COM; defaulting to no policy
I20260809 09:32:48.406543 27200 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.125.129@KRBTEST.COM'} at 127.20.125.129:43937
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.125.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.125.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.130.keytab.
Entry for principal kudu/127.20.125.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.130.keytab.
I20260809 09:32:48.426553 20982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
/tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786267941944493-20982-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786267941944493-20982-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786267941944493-20982-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786267941944493-20982-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.125.130:0
--local_ip_for_outbound_sockets=127.20.125.130
--webserver_interface=127.20.125.130
--webserver_port=0
--tserver_master_addrs=127.20.125.190:39399
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.130.keytab
--principal=kudu/127.20.125.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.125.148:40853
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786267941944493-20982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786267941944493-20982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786267941944493-20982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 09:32:48.541021 27508 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 09:32:48.541270 27508 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 09:32:48.541313 27508 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 09:32:48.545346 27508 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 09:32:48.545403 27508 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 09:32:48.545460 27508 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.125.130
I20260809 09:32:48.550679 27508 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.125.148:40853
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786267941944493-20982-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786267941944493-20982-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.130.keytab
--principal=kudu/127.20.125.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.125.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786267941944493-20982-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.125.130
--webserver_port=0
--tserver_master_addrs=127.20.125.190:39399
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27508
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.125.130
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786267941944493-20982-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 08:38:55 UTC on ccycloud.quasar-njjleu.root.comops.site
build id 4575935
I20260809 09:32:48.551746 27508 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 09:32:48.552850 27508 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 09:32:48.562495 27518 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:32:48.562512 27515 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:32:48.562738 27508 server_base.cc:1034] running on GCE node
W20260809 09:32:48.562489 27513 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:32:48.563182 27508 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 09:32:48.563815 27508 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 09:32:48.565017 27508 hybrid_clock.cc:648] HybridClock initialized: now 1786267968564999 us; error 36 us; skew 500 ppm
Aug 09 09:32:48 dist-test-slave-7kzw krb5kdc[27137](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786267968, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 09:32:48.568186 27508 init.cc:377] Logged in from keytab as kudu/127.20.125.130@KRBTEST.COM (short username kudu)
I20260809 09:32:48.568958 27508 fs_manager.cc:359] Metadata directory not provided
I20260809 09:32:48.569052 27508 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 09:32:48.569324 27508 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 09:32:48.571524 27508 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786267941944493-20982-0/minicluster-data/ts-1/data/instance:
uuid: "ef604f29695544c095a058123660345a"
format_stamp: "Formatted at 2026-08-09 09:32:48 on dist-test-slave-7kzw"
server_key: "62f946aba78504267d9a24bf6e29c0d7"
server_key_iv: "b461f6ae5d6cec87ccc8dd5ea748455a"
server_key_version: "encryptionkey@0"
I20260809 09:32:48.572086 27508 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786267941944493-20982-0/minicluster-data/ts-1/wal/instance:
uuid: "ef604f29695544c095a058123660345a"
format_stamp: "Formatted at 2026-08-09 09:32:48 on dist-test-slave-7kzw"
server_key: "62f946aba78504267d9a24bf6e29c0d7"
server_key_iv: "b461f6ae5d6cec87ccc8dd5ea748455a"
server_key_version: "encryptionkey@0"
I20260809 09:32:48.576018 27508 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260809 09:32:48.578435 27528 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 09:32:48.579759 27508 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 09:32:48.579898 27508 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786267941944493-20982-0/minicluster-data/ts-1/data,/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786267941944493-20982-0/minicluster-data/ts-1/wal
uuid: "ef604f29695544c095a058123660345a"
format_stamp: "Formatted at 2026-08-09 09:32:48 on dist-test-slave-7kzw"
server_key: "62f946aba78504267d9a24bf6e29c0d7"
server_key_iv: "b461f6ae5d6cec87ccc8dd5ea748455a"
server_key_version: "encryptionkey@0"
I20260809 09:32:48.580013 27508 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786267941944493-20982-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786267941944493-20982-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786267941944493-20982-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 09:32:48.628839 27508 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 09:32:48.632431 27508 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 09:32:48.632642 27508 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 09:32:48.633282 27508 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 09:32:48.634152 27508 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 09:32:48.634202 27508 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 09:32:48.634245 27508 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 09:32:48.634260 27508 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 09:32:48.643625 27508 rpc_server.cc:307] RPC server started. Bound to: 127.20.125.130:45523
I20260809 09:32:48.643647 27683 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.125.130:45523 every 8 connection(s)
I20260809 09:32:48.644240 27508 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786267941944493-20982-0/minicluster-data/ts-1/data/info.pb
I20260809 09:32:48.644713 20982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu as pid 27508
I20260809 09:32:48.644831 20982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786267941944493-20982-0/minicluster-data/ts-1/wal/instance
I20260809 09:32:48.645164 20982 external_mini_cluster.cc:1428] Setting key 48d36c818daf2e0c57b00e954403eafd
Aug 09 09:32:48 dist-test-slave-7kzw krb5kdc[27137](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786267968, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.130@KRBTEST.COM for kudu/127.20.125.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.20.125.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.125.131@KRBTEST.COM" created.
I20260809 09:32:48.661366 27262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:48.645999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.130:36017 (local address 127.20.125.190:39399)
0809 09:32:48.646153 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0809 09:32:48.646158 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 09:32:48.647861 (+ 1703us) server_negotiation.cc:407] Connection header received
0809 09:32:48.649456 (+ 1595us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:32:48.649461 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:32:48.649528 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:32:48.649633 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:32:48.650996 (+ 1363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:48.651868 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:48.653600 (+ 1732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:48.654038 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:48.654194 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:48.657039 (+ 2845us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:32:48.657045 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:32:48.657049 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:32:48.657085 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:32:48.658789 (+ 1704us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:32:48.659412 (+ 623us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:32:48.659415 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:32:48.659417 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:32:48.659533 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:32:48.659974 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:32:48.659981 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:32:48.659986 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:32:48.660212 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:32:48.660301 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0809 09:32:48.661062 (+ 761us) server_negotiation.cc:299] Negotiation successful
0809 09:32:48.661226 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260809 09:32:48.662554 27687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:48.646525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:39399 (local address 127.20.125.130:36017)
0809 09:32:48.647649 (+ 1124us) negotiation.cc:107] Waiting for socket to connect
0809 09:32:48.647727 (+ 78us) client_negotiation.cc:174] Beginning negotiation
0809 09:32:48.649072 (+ 1345us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:32:48.649842 (+ 770us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:32:48.649850 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:32:48.650247 (+ 397us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:32:48.650841 (+ 594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:32:48.650856 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:32:48.652041 (+ 1185us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:32:48.652045 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:32:48.653465 (+ 1420us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:32:48.653478 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:32:48.654244 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:32:48.654249 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:32:48.654448 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:48.655184 (+ 736us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 09:32:48.655210 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:32:48.656840 (+ 1630us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 09:32:48.658963 (+ 2123us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:32:48.658972 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:32:48.658996 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:32:48.659307 (+ 311us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:32:48.659657 (+ 350us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:32:48.659660 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:32:48.659662 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:32:48.659857 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:32:48.660335 (+ 478us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 09:32:48.660345 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 09:32:48.660720 (+ 375us) client_negotiation.cc:769] Sending connection context
0809 09:32:48.660959 (+ 239us) client_negotiation.cc:240] Negotiation successful
0809 09:32:48.661385 (+ 426us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":347,"thread_start_us":138,"threads_started":1}
I20260809 09:32:48.663889 27684 heartbeater.cc:347] Connected to a master server at 127.20.125.190:39399
I20260809 09:32:48.664184 27684 heartbeater.cc:464] Registering TS with master...
I20260809 09:32:48.664841 27684 heartbeater.cc:511] Master 127.20.125.190:39399 requested a full tablet report, sending...
I20260809 09:32:48.666178 27200 ts_manager.cc:194] Registered new tserver with Master: ef604f29695544c095a058123660345a (127.20.125.130:45523)
I20260809 09:32:48.666903 27200 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.125.130@KRBTEST.COM'} at 127.20.125.130:36017
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.125.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.131.keytab.
Entry for principal kudu/127.20.125.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.131.keytab.
I20260809 09:32:48.679885 20982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
/tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786267941944493-20982-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786267941944493-20982-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786267941944493-20982-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786267941944493-20982-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.125.131:0
--local_ip_for_outbound_sockets=127.20.125.131
--webserver_interface=127.20.125.131
--webserver_port=0
--tserver_master_addrs=127.20.125.190:39399
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.131.keytab
--principal=kudu/127.20.125.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.125.148:40853
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786267941944493-20982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786267941944493-20982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786267941944493-20982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 09:32:48.796870 27691 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 09:32:48.797118 27691 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 09:32:48.797163 27691 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 09:32:48.801393 27691 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 09:32:48.801458 27691 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 09:32:48.801542 27691 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.125.131
I20260809 09:32:48.806878 27691 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.125.148:40853
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786267941944493-20982-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786267941944493-20982-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.131.keytab
--principal=kudu/127.20.125.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.125.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786267941944493-20982-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.125.131
--webserver_port=0
--tserver_master_addrs=127.20.125.190:39399
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27691
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.125.131
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786267941944493-20982-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 08:38:55 UTC on ccycloud.quasar-njjleu.root.comops.site
build id 4575935
I20260809 09:32:48.807960 27691 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 09:32:48.808979 27691 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 09:32:48.818624 27698 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:32:48.818630 27702 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:32:48.818630 27696 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:32:48.819238 27691 server_base.cc:1034] running on GCE node
I20260809 09:32:48.819684 27691 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 09:32:48.820389 27691 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 09:32:48.821591 27691 hybrid_clock.cc:648] HybridClock initialized: now 1786267968821568 us; error 35 us; skew 500 ppm
Aug 09 09:32:48 dist-test-slave-7kzw krb5kdc[27137](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786267968, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 09:32:48.824756 27691 init.cc:377] Logged in from keytab as kudu/127.20.125.131@KRBTEST.COM (short username kudu)
I20260809 09:32:48.825519 27691 fs_manager.cc:359] Metadata directory not provided
I20260809 09:32:48.825593 27691 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 09:32:48.825771 27691 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 09:32:48.828081 27691 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786267941944493-20982-0/minicluster-data/ts-2/data/instance:
uuid: "a01f9cbd815f40a19aa95a39fccd3ecf"
format_stamp: "Formatted at 2026-08-09 09:32:48 on dist-test-slave-7kzw"
server_key: "8128fea7ed314c58a2f4efa0cd228901"
server_key_iv: "f873451f3228f593c89e86338c49090c"
server_key_version: "encryptionkey@0"
I20260809 09:32:48.828663 27691 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786267941944493-20982-0/minicluster-data/ts-2/wal/instance:
uuid: "a01f9cbd815f40a19aa95a39fccd3ecf"
format_stamp: "Formatted at 2026-08-09 09:32:48 on dist-test-slave-7kzw"
server_key: "8128fea7ed314c58a2f4efa0cd228901"
server_key_iv: "f873451f3228f593c89e86338c49090c"
server_key_version: "encryptionkey@0"
I20260809 09:32:48.832741 27691 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 09:32:48.835286 27709 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 09:32:48.836614 27691 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 09:32:48.836766 27691 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786267941944493-20982-0/minicluster-data/ts-2/data,/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786267941944493-20982-0/minicluster-data/ts-2/wal
uuid: "a01f9cbd815f40a19aa95a39fccd3ecf"
format_stamp: "Formatted at 2026-08-09 09:32:48 on dist-test-slave-7kzw"
server_key: "8128fea7ed314c58a2f4efa0cd228901"
server_key_iv: "f873451f3228f593c89e86338c49090c"
server_key_version: "encryptionkey@0"
I20260809 09:32:48.836886 27691 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786267941944493-20982-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786267941944493-20982-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786267941944493-20982-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 09:32:48.884402 27691 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 09:32:48.888278 27691 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 09:32:48.888451 27691 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 09:32:48.889038 27691 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 09:32:48.889917 27691 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 09:32:48.889972 27691 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 09:32:48.890017 27691 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 09:32:48.890034 27691 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 09:32:48.899770 27691 rpc_server.cc:307] RPC server started. Bound to: 127.20.125.131:43009
I20260809 09:32:48.899794 27874 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.125.131:43009 every 8 connection(s)
I20260809 09:32:48.900538 27691 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786267941944493-20982-0/minicluster-data/ts-2/data/info.pb
I20260809 09:32:48.908363 20982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu as pid 27691
I20260809 09:32:48.908541 20982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786267941944493-20982-0/minicluster-data/ts-2/wal/instance
I20260809 09:32:48.908924 20982 external_mini_cluster.cc:1428] Setting key ab02d48dc71b667288dec58ae708a32b
Aug 09 09:32:48 dist-test-slave-7kzw krb5kdc[27137](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786267968, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.131@KRBTEST.COM for kudu/127.20.125.190@KRBTEST.COM
I20260809 09:32:48.915838 27262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:48.902505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.131:44835 (local address 127.20.125.190:39399)
0809 09:32:48.902676 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0809 09:32:48.902682 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 09:32:48.904018 (+ 1336us) server_negotiation.cc:407] Connection header received
0809 09:32:48.904948 (+ 930us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:32:48.904951 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:32:48.905002 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:32:48.905087 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:32:48.906505 (+ 1418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:48.907089 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:48.908411 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:48.908803 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:48.908931 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:48.912024 (+ 3093us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:32:48.912027 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:32:48.912029 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:32:48.912057 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:32:48.913714 (+ 1657us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:32:48.914222 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:32:48.914225 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:32:48.914226 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:32:48.914313 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:32:48.914740 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:32:48.914743 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:32:48.914744 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:32:48.914935 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:32:48.915023 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0809 09:32:48.915578 (+ 555us) server_negotiation.cc:299] Negotiation successful
0809 09:32:48.915703 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260809 09:32:48.916725 27877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:48.902931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:39399 (local address 127.20.125.131:44835)
0809 09:32:48.903862 (+ 931us) negotiation.cc:107] Waiting for socket to connect
0809 09:32:48.903906 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 09:32:48.904754 (+ 848us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:32:48.905328 (+ 574us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:32:48.905335 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:32:48.905762 (+ 427us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:32:48.906326 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:32:48.906341 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:32:48.907235 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:32:48.907239 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:32:48.908258 (+ 1019us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:32:48.908270 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:32:48.908967 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:32:48.908990 (+ 23us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:32:48.909193 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:48.909932 (+ 739us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 09:32:48.909955 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:32:48.911839 (+ 1884us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 09:32:48.913839 (+ 2000us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:32:48.913844 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:32:48.913857 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:32:48.914120 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:32:48.914397 (+ 277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:32:48.914399 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:32:48.914400 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:32:48.914625 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:32:48.915060 (+ 435us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 09:32:48.915066 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 09:32:48.915325 (+ 259us) client_negotiation.cc:769] Sending connection context
0809 09:32:48.915495 (+ 170us) client_negotiation.cc:240] Negotiation successful
0809 09:32:48.915804 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":329,"thread_start_us":127,"threads_started":1}
I20260809 09:32:48.917959 27875 heartbeater.cc:347] Connected to a master server at 127.20.125.190:39399
I20260809 09:32:48.918211 27875 heartbeater.cc:464] Registering TS with master...
I20260809 09:32:48.918761 27875 heartbeater.cc:511] Master 127.20.125.190:39399 requested a full tablet report, sending...
I20260809 09:32:48.919793 27200 ts_manager.cc:194] Registered new tserver with Master: a01f9cbd815f40a19aa95a39fccd3ecf (127.20.125.131:43009)
I20260809 09:32:48.920343 27200 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.125.131@KRBTEST.COM'} at 127.20.125.131:44835
I20260809 09:32:48.923002 20982 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 09:32:48.932799 27262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:48.925058 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42662 (local address 127.20.125.190:39399)
0809 09:32:48.925259 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0809 09:32:48.925264 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 09:32:48.925394 (+ 130us) server_negotiation.cc:407] Connection header received
0809 09:32:48.925561 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:32:48.925565 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:32:48.925612 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:32:48.925696 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:32:48.926511 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:48.927052 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:48.928096 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:48.928326 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:48.928439 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:48.929786 (+ 1347us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:32:48.929789 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:32:48.929791 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:32:48.929813 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:32:48.931135 (+ 1322us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:32:48.931577 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:32:48.931580 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:32:48.931581 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:32:48.931656 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:32:48.931933 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:32:48.931938 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:32:48.931939 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:32:48.932149 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:32:48.932292 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0809 09:32:48.932524 (+ 232us) server_negotiation.cc:299] Negotiation successful
0809 09:32:48.932637 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260809 09:32:48.935436 27200 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42662:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 09:32:48.938405 27200 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 09:32:48.952807 27890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:48.946872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.130:45523 (local address 127.0.0.1:54494)
0809 09:32:48.947581 (+ 709us) negotiation.cc:107] Waiting for socket to connect
0809 09:32:48.947599 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 09:32:48.947692 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:32:48.948471 (+ 779us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:32:48.948478 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:32:48.948501 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 09:32:48.948648 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:32:48.948654 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:32:48.949839 (+ 1185us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:32:48.949845 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:32:48.951270 (+ 1425us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:32:48.951281 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:32:48.952266 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:32:48.952271 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:32:48.952376 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:48.952416 (+ 40us) client_negotiation.cc:769] Sending connection context
0809 09:32:48.952520 (+ 104us) client_negotiation.cc:240] Negotiation successful
0809 09:32:48.952643 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":641,"mutex_wait_us":50,"thread_start_us":90,"threads_started":1}
I20260809 09:32:48.952903 27895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:48.947519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54494 (local address 127.20.125.130:45523)
0809 09:32:48.947944 (+ 425us) server_negotiation.cc:206] Beginning negotiation
0809 09:32:48.947958 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 09:32:48.947981 (+ 23us) server_negotiation.cc:407] Connection header received
0809 09:32:48.948063 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:32:48.948069 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:32:48.948290 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:32:48.948815 (+ 525us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 09:32:48.948979 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:48.949713 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:48.951382 (+ 1669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:48.952116 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:48.952306 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:48.952493 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0809 09:32:48.952549 (+ 56us) server_negotiation.cc:299] Negotiation successful
0809 09:32:48.952707 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":321,"thread_start_us":97,"threads_started":1}
I20260809 09:32:48.953096 27889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:48.946781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.129:41219 (local address 127.0.0.1:60078)
0809 09:32:48.947364 (+ 583us) negotiation.cc:107] Waiting for socket to connect
0809 09:32:48.947417 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0809 09:32:48.947537 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:32:48.948829 (+ 1292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:32:48.948832 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:32:48.948849 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 09:32:48.948957 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:32:48.948963 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:32:48.950408 (+ 1445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:32:48.950411 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:32:48.951581 (+ 1170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:32:48.951594 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:32:48.952641 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:32:48.952645 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:32:48.952787 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:48.952817 (+ 30us) client_negotiation.cc:769] Sending connection context
0809 09:32:48.952869 (+ 52us) client_negotiation.cc:240] Negotiation successful
0809 09:32:48.952949 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":407,"mutex_wait_us":19,"thread_start_us":88,"threads_started":1}
I20260809 09:32:48.953094 27892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:48.947481 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.131:43009 (local address 127.0.0.1:36520)
0809 09:32:48.948068 (+ 587us) negotiation.cc:107] Waiting for socket to connect
0809 09:32:48.948082 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 09:32:48.948168 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:32:48.948777 (+ 609us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:32:48.948780 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:32:48.948798 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 09:32:48.948886 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:32:48.948892 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:32:48.950262 (+ 1370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:32:48.950266 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:32:48.951476 (+ 1210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:32:48.951483 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:32:48.952754 (+ 1271us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:32:48.952757 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:32:48.952869 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:48.952888 (+ 19us) client_negotiation.cc:769] Sending connection context
0809 09:32:48.952949 (+ 61us) client_negotiation.cc:240] Negotiation successful
0809 09:32:48.952997 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":511,"mutex_wait_us":59,"thread_start_us":97,"threads_started":1}
I20260809 09:32:48.953663 27891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:48.946939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60078 (local address 127.20.125.129:41219)
0809 09:32:48.948445 (+ 1506us) server_negotiation.cc:206] Beginning negotiation
0809 09:32:48.948451 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 09:32:48.948470 (+ 19us) server_negotiation.cc:407] Connection header received
0809 09:32:48.948533 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:32:48.948537 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:32:48.948683 (+ 146us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:32:48.949103 (+ 420us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 09:32:48.949237 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:48.950303 (+ 1066us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:48.951689 (+ 1386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:48.952513 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:48.953226 (+ 713us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:48.953369 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0809 09:32:48.953427 (+ 58us) server_negotiation.cc:299] Negotiation successful
0809 09:32:48.953539 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1382,"thread_start_us":64,"threads_started":1}
I20260809 09:32:48.953701 27897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:48.947882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36520 (local address 127.20.125.131:43009)
0809 09:32:48.948332 (+ 450us) server_negotiation.cc:206] Beginning negotiation
0809 09:32:48.948346 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 09:32:48.948370 (+ 24us) server_negotiation.cc:407] Connection header received
0809 09:32:48.948455 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:32:48.948462 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:32:48.948634 (+ 172us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:32:48.948813 (+ 179us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 09:32:48.948967 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:48.950112 (+ 1145us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:48.951590 (+ 1478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:48.952407 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:48.952543 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:48.952696 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0809 09:32:48.953451 (+ 755us) server_negotiation.cc:299] Negotiation successful
0809 09:32:48.953554 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":353,"thread_start_us":148,"threads_started":1}
I20260809 09:32:48.956130 27590 tablet_service.cc:1467] Processing CreateTablet for tablet 8623e355aa494e79a8e13554b24d857d (DEFAULT_TABLE table=test-table [id=aad70c506c344718934faefc10d4cc39]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 09:32:48.956382 27398 tablet_service.cc:1467] Processing CreateTablet for tablet 8623e355aa494e79a8e13554b24d857d (DEFAULT_TABLE table=test-table [id=aad70c506c344718934faefc10d4cc39]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 09:32:48.956383 27780 tablet_service.cc:1467] Processing CreateTablet for tablet 8623e355aa494e79a8e13554b24d857d (DEFAULT_TABLE table=test-table [id=aad70c506c344718934faefc10d4cc39]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 09:32:48.957207 27590 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8623e355aa494e79a8e13554b24d857d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 09:32:48.957396 27398 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8623e355aa494e79a8e13554b24d857d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 09:32:48.957402 27780 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8623e355aa494e79a8e13554b24d857d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 09:32:48.964161 27899 tablet_bootstrap.cc:492] T 8623e355aa494e79a8e13554b24d857d P ef604f29695544c095a058123660345a: Bootstrap starting.
I20260809 09:32:48.965308 27904 tablet_bootstrap.cc:492] T 8623e355aa494e79a8e13554b24d857d P a01f9cbd815f40a19aa95a39fccd3ecf: Bootstrap starting.
I20260809 09:32:48.966414 27899 tablet_bootstrap.cc:654] T 8623e355aa494e79a8e13554b24d857d P ef604f29695544c095a058123660345a: Neither blocks nor log segments found. Creating new log.
I20260809 09:32:48.966812 27906 tablet_bootstrap.cc:492] T 8623e355aa494e79a8e13554b24d857d P 6f03935c0ae54f6c8c48865f4b870ef8: Bootstrap starting.
I20260809 09:32:48.967191 27904 tablet_bootstrap.cc:654] T 8623e355aa494e79a8e13554b24d857d P a01f9cbd815f40a19aa95a39fccd3ecf: Neither blocks nor log segments found. Creating new log.
I20260809 09:32:48.967577 27899 log.cc:824] T 8623e355aa494e79a8e13554b24d857d P ef604f29695544c095a058123660345a: Log is configured to *not* fsync() on all Append() calls
I20260809 09:32:48.968397 27904 log.cc:824] T 8623e355aa494e79a8e13554b24d857d P a01f9cbd815f40a19aa95a39fccd3ecf: Log is configured to *not* fsync() on all Append() calls
I20260809 09:32:48.969044 27906 tablet_bootstrap.cc:654] T 8623e355aa494e79a8e13554b24d857d P 6f03935c0ae54f6c8c48865f4b870ef8: Neither blocks nor log segments found. Creating new log.
I20260809 09:32:48.969743 27899 tablet_bootstrap.cc:492] T 8623e355aa494e79a8e13554b24d857d P ef604f29695544c095a058123660345a: No bootstrap required, opened a new log
I20260809 09:32:48.969906 27906 log.cc:824] T 8623e355aa494e79a8e13554b24d857d P 6f03935c0ae54f6c8c48865f4b870ef8: Log is configured to *not* fsync() on all Append() calls
I20260809 09:32:48.969954 27899 ts_tablet_manager.cc:1397] T 8623e355aa494e79a8e13554b24d857d P ef604f29695544c095a058123660345a: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260809 09:32:48.970428 27904 tablet_bootstrap.cc:492] T 8623e355aa494e79a8e13554b24d857d P a01f9cbd815f40a19aa95a39fccd3ecf: No bootstrap required, opened a new log
I20260809 09:32:48.970654 27904 ts_tablet_manager.cc:1397] T 8623e355aa494e79a8e13554b24d857d P a01f9cbd815f40a19aa95a39fccd3ecf: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.001s
I20260809 09:32:48.972046 27906 tablet_bootstrap.cc:492] T 8623e355aa494e79a8e13554b24d857d P 6f03935c0ae54f6c8c48865f4b870ef8: No bootstrap required, opened a new log
I20260809 09:32:48.972235 27906 ts_tablet_manager.cc:1397] T 8623e355aa494e79a8e13554b24d857d P 6f03935c0ae54f6c8c48865f4b870ef8: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260809 09:32:48.973197 27899 raft_consensus.cc:348] T 8623e355aa494e79a8e13554b24d857d P ef604f29695544c095a058123660345a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f03935c0ae54f6c8c48865f4b870ef8" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 41219 } } peers { permanent_uuid: "ef604f29695544c095a058123660345a" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 45523 } } peers { permanent_uuid: "a01f9cbd815f40a19aa95a39fccd3ecf" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 43009 } }
I20260809 09:32:48.973475 27899 raft_consensus.cc:374] T 8623e355aa494e79a8e13554b24d857d P ef604f29695544c095a058123660345a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 09:32:48.973565 27899 raft_consensus.cc:729] T 8623e355aa494e79a8e13554b24d857d P ef604f29695544c095a058123660345a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef604f29695544c095a058123660345a, State: Initialized, Role: FOLLOWER
I20260809 09:32:48.974050 27899 consensus_queue.cc:260] T 8623e355aa494e79a8e13554b24d857d P ef604f29695544c095a058123660345a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f03935c0ae54f6c8c48865f4b870ef8" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 41219 } } peers { permanent_uuid: "ef604f29695544c095a058123660345a" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 45523 } } peers { permanent_uuid: "a01f9cbd815f40a19aa95a39fccd3ecf" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 43009 } }
I20260809 09:32:48.974802 27684 heartbeater.cc:503] Master 127.20.125.190:39399 was elected leader, sending a full tablet report...
I20260809 09:32:48.975088 27899 ts_tablet_manager.cc:1428] T 8623e355aa494e79a8e13554b24d857d P ef604f29695544c095a058123660345a: Time spent starting tablet: real 0.005s user 0.005s sys 0.001s
I20260809 09:32:48.975330 27904 raft_consensus.cc:348] T 8623e355aa494e79a8e13554b24d857d P a01f9cbd815f40a19aa95a39fccd3ecf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f03935c0ae54f6c8c48865f4b870ef8" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 41219 } } peers { permanent_uuid: "ef604f29695544c095a058123660345a" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 45523 } } peers { permanent_uuid: "a01f9cbd815f40a19aa95a39fccd3ecf" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 43009 } }
I20260809 09:32:48.975589 27904 raft_consensus.cc:374] T 8623e355aa494e79a8e13554b24d857d P a01f9cbd815f40a19aa95a39fccd3ecf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 09:32:48.975636 27904 raft_consensus.cc:729] T 8623e355aa494e79a8e13554b24d857d P a01f9cbd815f40a19aa95a39fccd3ecf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a01f9cbd815f40a19aa95a39fccd3ecf, State: Initialized, Role: FOLLOWER
I20260809 09:32:48.976048 27904 consensus_queue.cc:260] T 8623e355aa494e79a8e13554b24d857d P a01f9cbd815f40a19aa95a39fccd3ecf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f03935c0ae54f6c8c48865f4b870ef8" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 41219 } } peers { permanent_uuid: "ef604f29695544c095a058123660345a" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 45523 } } peers { permanent_uuid: "a01f9cbd815f40a19aa95a39fccd3ecf" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 43009 } }
I20260809 09:32:48.976641 27906 raft_consensus.cc:348] T 8623e355aa494e79a8e13554b24d857d P 6f03935c0ae54f6c8c48865f4b870ef8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f03935c0ae54f6c8c48865f4b870ef8" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 41219 } } peers { permanent_uuid: "ef604f29695544c095a058123660345a" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 45523 } } peers { permanent_uuid: "a01f9cbd815f40a19aa95a39fccd3ecf" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 43009 } }
I20260809 09:32:48.976920 27906 raft_consensus.cc:374] T 8623e355aa494e79a8e13554b24d857d P 6f03935c0ae54f6c8c48865f4b870ef8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 09:32:48.977021 27906 raft_consensus.cc:729] T 8623e355aa494e79a8e13554b24d857d P 6f03935c0ae54f6c8c48865f4b870ef8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f03935c0ae54f6c8c48865f4b870ef8, State: Initialized, Role: FOLLOWER
I20260809 09:32:48.977101 27904 ts_tablet_manager.cc:1428] T 8623e355aa494e79a8e13554b24d857d P a01f9cbd815f40a19aa95a39fccd3ecf: Time spent starting tablet: real 0.006s user 0.003s sys 0.002s
I20260809 09:32:48.977489 27875 heartbeater.cc:503] Master 127.20.125.190:39399 was elected leader, sending a full tablet report...
I20260809 09:32:48.977515 27906 consensus_queue.cc:260] T 8623e355aa494e79a8e13554b24d857d P 6f03935c0ae54f6c8c48865f4b870ef8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f03935c0ae54f6c8c48865f4b870ef8" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 41219 } } peers { permanent_uuid: "ef604f29695544c095a058123660345a" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 45523 } } peers { permanent_uuid: "a01f9cbd815f40a19aa95a39fccd3ecf" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 43009 } }
I20260809 09:32:48.978221 27499 heartbeater.cc:503] Master 127.20.125.190:39399 was elected leader, sending a full tablet report...
I20260809 09:32:48.979360 27906 ts_tablet_manager.cc:1428] T 8623e355aa494e79a8e13554b24d857d P 6f03935c0ae54f6c8c48865f4b870ef8: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260809 09:32:49.070658 27914 raft_consensus.cc:482] T 8623e355aa494e79a8e13554b24d857d P 6f03935c0ae54f6c8c48865f4b870ef8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 09:32:49.070931 27914 raft_consensus.cc:504] T 8623e355aa494e79a8e13554b24d857d P 6f03935c0ae54f6c8c48865f4b870ef8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f03935c0ae54f6c8c48865f4b870ef8" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 41219 } } peers { permanent_uuid: "ef604f29695544c095a058123660345a" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 45523 } } peers { permanent_uuid: "a01f9cbd815f40a19aa95a39fccd3ecf" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 43009 } }
I20260809 09:32:49.072130 27914 leader_election.cc:288] T 8623e355aa494e79a8e13554b24d857d P 6f03935c0ae54f6c8c48865f4b870ef8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ef604f29695544c095a058123660345a (127.20.125.130:45523), a01f9cbd815f40a19aa95a39fccd3ecf (127.20.125.131:43009)
I20260809 09:32:49.077262 27916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:49.072484 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.131:43009 (local address 127.20.125.129:36435)
0809 09:32:49.073027 (+ 543us) negotiation.cc:107] Waiting for socket to connect
0809 09:32:49.073041 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 09:32:49.073131 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:32:49.073393 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:32:49.073396 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:32:49.073443 (+ 47us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 09:32:49.073701 (+ 258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:32:49.073708 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:32:49.074961 (+ 1253us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:32:49.074964 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:32:49.075959 (+ 995us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:32:49.075970 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:32:49.076817 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:32:49.076820 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:32:49.076921 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:49.076947 (+ 26us) client_negotiation.cc:769] Sending connection context
0809 09:32:49.077017 (+ 70us) client_negotiation.cc:240] Negotiation successful
0809 09:32:49.077089 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":481,"mutex_wait_us":207,"spinlock_wait_cycles":7808,"thread_start_us":129,"threads_started":1}
I20260809 09:32:49.077291 27897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:49.072794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.129:36435 (local address 127.20.125.131:43009)
0809 09:32:49.072931 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0809 09:32:49.072937 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 09:32:49.073122 (+ 185us) server_negotiation.cc:407] Connection header received
0809 09:32:49.073215 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:32:49.073219 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:32:49.073274 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:32:49.073368 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 09:32:49.073848 (+ 480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:49.074797 (+ 949us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:49.076102 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:49.076689 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:49.076817 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:49.076874 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0809 09:32:49.077055 (+ 181us) server_negotiation.cc:299] Negotiation successful
0809 09:32:49.077150 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260809 09:32:49.077826 27915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:49.072484 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.130:45523 (local address 127.20.125.129:57435)
0809 09:32:49.072921 (+ 437us) negotiation.cc:107] Waiting for socket to connect
0809 09:32:49.072941 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 09:32:49.073085 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:32:49.073393 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:32:49.073396 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:32:49.073441 (+ 45us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 09:32:49.073701 (+ 260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:32:49.073708 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:32:49.074932 (+ 1224us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:32:49.074935 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:32:49.076283 (+ 1348us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:32:49.076295 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:32:49.077320 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:32:49.077325 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:32:49.077438 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:49.077472 (+ 34us) client_negotiation.cc:769] Sending connection context
0809 09:32:49.077586 (+ 114us) client_negotiation.cc:240] Negotiation successful
0809 09:32:49.077662 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":353,"mutex_wait_us":122,"spinlock_wait_cycles":39168,"thread_start_us":160,"threads_started":1}
I20260809 09:32:49.077807 27895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:49.072685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.129:57435 (local address 127.20.125.130:45523)
0809 09:32:49.072933 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0809 09:32:49.072938 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 09:32:49.073039 (+ 101us) server_negotiation.cc:407] Connection header received
0809 09:32:49.073216 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:32:49.073219 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:32:49.073274 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:32:49.073368 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 09:32:49.073861 (+ 493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:49.074783 (+ 922us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:49.076426 (+ 1643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:49.077067 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:49.077201 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:49.077248 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0809 09:32:49.077614 (+ 366us) server_negotiation.cc:299] Negotiation successful
0809 09:32:49.077693 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158}
I20260809 09:32:49.078226 27807 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8623e355aa494e79a8e13554b24d857d" candidate_uuid: "6f03935c0ae54f6c8c48865f4b870ef8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a01f9cbd815f40a19aa95a39fccd3ecf" is_pre_election: true
I20260809 09:32:49.078580 27807 raft_consensus.cc:2454] T 8623e355aa494e79a8e13554b24d857d P a01f9cbd815f40a19aa95a39fccd3ecf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f03935c0ae54f6c8c48865f4b870ef8 in term 0.
I20260809 09:32:49.078811 27618 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8623e355aa494e79a8e13554b24d857d" candidate_uuid: "6f03935c0ae54f6c8c48865f4b870ef8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef604f29695544c095a058123660345a" is_pre_election: true
I20260809 09:32:49.079126 27618 raft_consensus.cc:2454] T 8623e355aa494e79a8e13554b24d857d P ef604f29695544c095a058123660345a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f03935c0ae54f6c8c48865f4b870ef8 in term 0.
I20260809 09:32:49.079124 27319 leader_election.cc:302] T 8623e355aa494e79a8e13554b24d857d P 6f03935c0ae54f6c8c48865f4b870ef8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f03935c0ae54f6c8c48865f4b870ef8, a01f9cbd815f40a19aa95a39fccd3ecf; no voters:
I20260809 09:32:49.079420 27914 raft_consensus.cc:2781] T 8623e355aa494e79a8e13554b24d857d P 6f03935c0ae54f6c8c48865f4b870ef8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 09:32:49.079538 27914 raft_consensus.cc:482] T 8623e355aa494e79a8e13554b24d857d P 6f03935c0ae54f6c8c48865f4b870ef8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 09:32:49.079615 27914 raft_consensus.cc:3037] T 8623e355aa494e79a8e13554b24d857d P 6f03935c0ae54f6c8c48865f4b870ef8 [term 0 FOLLOWER]: Advancing to term 1
I20260809 09:32:49.080821 27914 raft_consensus.cc:504] T 8623e355aa494e79a8e13554b24d857d P 6f03935c0ae54f6c8c48865f4b870ef8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f03935c0ae54f6c8c48865f4b870ef8" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 41219 } } peers { permanent_uuid: "ef604f29695544c095a058123660345a" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 45523 } } peers { permanent_uuid: "a01f9cbd815f40a19aa95a39fccd3ecf" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 43009 } }
I20260809 09:32:49.081245 27914 leader_election.cc:288] T 8623e355aa494e79a8e13554b24d857d P 6f03935c0ae54f6c8c48865f4b870ef8 [CANDIDATE]: Term 1 election: Requested vote from peers ef604f29695544c095a058123660345a (127.20.125.130:45523), a01f9cbd815f40a19aa95a39fccd3ecf (127.20.125.131:43009)
I20260809 09:32:49.081606 27807 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8623e355aa494e79a8e13554b24d857d" candidate_uuid: "6f03935c0ae54f6c8c48865f4b870ef8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a01f9cbd815f40a19aa95a39fccd3ecf"
I20260809 09:32:49.081725 27807 raft_consensus.cc:3037] T 8623e355aa494e79a8e13554b24d857d P a01f9cbd815f40a19aa95a39fccd3ecf [term 0 FOLLOWER]: Advancing to term 1
I20260809 09:32:49.081743 27618 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8623e355aa494e79a8e13554b24d857d" candidate_uuid: "6f03935c0ae54f6c8c48865f4b870ef8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef604f29695544c095a058123660345a"
I20260809 09:32:49.081866 27618 raft_consensus.cc:3037] T 8623e355aa494e79a8e13554b24d857d P ef604f29695544c095a058123660345a [term 0 FOLLOWER]: Advancing to term 1
I20260809 09:32:49.082978 27618 raft_consensus.cc:2454] T 8623e355aa494e79a8e13554b24d857d P ef604f29695544c095a058123660345a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f03935c0ae54f6c8c48865f4b870ef8 in term 1.
I20260809 09:32:49.083077 27807 raft_consensus.cc:2454] T 8623e355aa494e79a8e13554b24d857d P a01f9cbd815f40a19aa95a39fccd3ecf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f03935c0ae54f6c8c48865f4b870ef8 in term 1.
I20260809 09:32:49.083298 27320 leader_election.cc:302] T 8623e355aa494e79a8e13554b24d857d P 6f03935c0ae54f6c8c48865f4b870ef8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f03935c0ae54f6c8c48865f4b870ef8, ef604f29695544c095a058123660345a; no voters:
I20260809 09:32:49.083551 27914 raft_consensus.cc:2781] T 8623e355aa494e79a8e13554b24d857d P 6f03935c0ae54f6c8c48865f4b870ef8 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 09:32:49.083832 27914 raft_consensus.cc:686] T 8623e355aa494e79a8e13554b24d857d P 6f03935c0ae54f6c8c48865f4b870ef8 [term 1 LEADER]: Becoming Leader. State: Replica: 6f03935c0ae54f6c8c48865f4b870ef8, State: Running, Role: LEADER
I20260809 09:32:49.084182 27914 consensus_queue.cc:237] T 8623e355aa494e79a8e13554b24d857d P 6f03935c0ae54f6c8c48865f4b870ef8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f03935c0ae54f6c8c48865f4b870ef8" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 41219 } } peers { permanent_uuid: "ef604f29695544c095a058123660345a" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 45523 } } peers { permanent_uuid: "a01f9cbd815f40a19aa95a39fccd3ecf" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 43009 } }
I20260809 09:32:49.087661 27200 catalog_manager.cc:5495] T 8623e355aa494e79a8e13554b24d857d P 6f03935c0ae54f6c8c48865f4b870ef8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6f03935c0ae54f6c8c48865f4b870ef8 (127.20.125.129). New cstate: current_term: 1 leader_uuid: "6f03935c0ae54f6c8c48865f4b870ef8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f03935c0ae54f6c8c48865f4b870ef8" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 41219 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ef604f29695544c095a058123660345a" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 45523 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a01f9cbd815f40a19aa95a39fccd3ecf" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 43009 } health_report { overall_health: UNKNOWN } } }
I20260809 09:32:49.107228 27891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:49.103227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60088 (local address 127.20.125.129:41219)
0809 09:32:49.103361 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0809 09:32:49.103365 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 09:32:49.103382 (+ 17us) server_negotiation.cc:407] Connection header received
0809 09:32:49.103506 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:32:49.103509 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:32:49.103606 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:32:49.103709 (+ 103us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 09:32:49.104051 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:49.104676 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:49.105617 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:49.105868 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:49.105974 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:49.106300 (+ 326us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 09:32:49.106823 (+ 523us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 09:32:49.106932 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0809 09:32:49.107039 (+ 107us) server_negotiation.cc:299] Negotiation successful
0809 09:32:49.107104 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260809 09:32:49.113832 27807 raft_consensus.cc:1261] T 8623e355aa494e79a8e13554b24d857d P a01f9cbd815f40a19aa95a39fccd3ecf [term 1 FOLLOWER]: Refusing update from remote peer 6f03935c0ae54f6c8c48865f4b870ef8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 09:32:49.113829 27618 raft_consensus.cc:1261] T 8623e355aa494e79a8e13554b24d857d P ef604f29695544c095a058123660345a [term 1 FOLLOWER]: Refusing update from remote peer 6f03935c0ae54f6c8c48865f4b870ef8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 09:32:49.114567 27914 consensus_queue.cc:1035] T 8623e355aa494e79a8e13554b24d857d P 6f03935c0ae54f6c8c48865f4b870ef8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef604f29695544c095a058123660345a" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 45523 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 09:32:49.114825 27918 consensus_queue.cc:1035] T 8623e355aa494e79a8e13554b24d857d P 6f03935c0ae54f6c8c48865f4b870ef8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a01f9cbd815f40a19aa95a39fccd3ecf" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 09:32:49.133665 27207 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42662:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 09:32:49.133890 27207 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42662:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 09:32:49.136853 27207 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 715e779361714d09a66e9084cc8b8767: Sending DeleteTablet for 3 replicas of tablet 8623e355aa494e79a8e13554b24d857d
I20260809 09:32:49.137660 27398 tablet_service.cc:1514] Processing DeleteTablet for tablet 8623e355aa494e79a8e13554b24d857d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 09:32:49 UTC) from {username='kudu', principal='kudu/127.20.125.190@KRBTEST.COM'} at 127.0.0.1:60078
I20260809 09:32:49.137737 27590 tablet_service.cc:1514] Processing DeleteTablet for tablet 8623e355aa494e79a8e13554b24d857d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 09:32:49 UTC) from {username='kudu', principal='kudu/127.20.125.190@KRBTEST.COM'} at 127.0.0.1:54494
I20260809 09:32:49.137859 27780 tablet_service.cc:1514] Processing DeleteTablet for tablet 8623e355aa494e79a8e13554b24d857d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 09:32:49 UTC) from {username='kudu', principal='kudu/127.20.125.190@KRBTEST.COM'} at 127.0.0.1:36520
I20260809 09:32:49.138510 27939 tablet_replica.cc:330] T 8623e355aa494e79a8e13554b24d857d P a01f9cbd815f40a19aa95a39fccd3ecf: stopping tablet replica
I20260809 09:32:49.138521 27938 tablet_replica.cc:330] T 8623e355aa494e79a8e13554b24d857d P ef604f29695544c095a058123660345a: stopping tablet replica
I20260809 09:32:49.138862 27938 raft_consensus.cc:2229] T 8623e355aa494e79a8e13554b24d857d P ef604f29695544c095a058123660345a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 09:32:49.138893 27939 raft_consensus.cc:2229] T 8623e355aa494e79a8e13554b24d857d P a01f9cbd815f40a19aa95a39fccd3ecf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 09:32:49.139036 20982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu with pid 27298
I20260809 09:32:49.139221 27939 raft_consensus.cc:2258] T 8623e355aa494e79a8e13554b24d857d P a01f9cbd815f40a19aa95a39fccd3ecf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 09:32:49.144490 27939 ts_tablet_manager.cc:1905] T 8623e355aa494e79a8e13554b24d857d P a01f9cbd815f40a19aa95a39fccd3ecf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 09:32:49.146170 27938 raft_consensus.cc:2258] T 8623e355aa494e79a8e13554b24d857d P ef604f29695544c095a058123660345a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 09:32:49.148054 27939 ts_tablet_manager.cc:1918] T 8623e355aa494e79a8e13554b24d857d P a01f9cbd815f40a19aa95a39fccd3ecf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 09:32:49.148095 27938 ts_tablet_manager.cc:1905] T 8623e355aa494e79a8e13554b24d857d P ef604f29695544c095a058123660345a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 09:32:49.148169 27939 log.cc:1196] T 8623e355aa494e79a8e13554b24d857d P a01f9cbd815f40a19aa95a39fccd3ecf: Deleting WAL directory at /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786267941944493-20982-0/minicluster-data/ts-2/wal/wals/8623e355aa494e79a8e13554b24d857d
I20260809 09:32:49.148576 27939 ts_tablet_manager.cc:1939] T 8623e355aa494e79a8e13554b24d857d P a01f9cbd815f40a19aa95a39fccd3ecf: Deleting consensus metadata
I20260809 09:32:49.150087 27181 catalog_manager.cc:4841] TS a01f9cbd815f40a19aa95a39fccd3ecf (127.20.125.131:43009): tablet 8623e355aa494e79a8e13554b24d857d (table test-table [id=aad70c506c344718934faefc10d4cc39]) successfully deleted
I20260809 09:32:49.151659 27938 ts_tablet_manager.cc:1918] T 8623e355aa494e79a8e13554b24d857d P ef604f29695544c095a058123660345a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 09:32:49.151767 27938 log.cc:1196] T 8623e355aa494e79a8e13554b24d857d P ef604f29695544c095a058123660345a: Deleting WAL directory at /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786267941944493-20982-0/minicluster-data/ts-1/wal/wals/8623e355aa494e79a8e13554b24d857d
I20260809 09:32:49.152094 27938 ts_tablet_manager.cc:1939] T 8623e355aa494e79a8e13554b24d857d P ef604f29695544c095a058123660345a: Deleting consensus metadata
I20260809 09:32:49.153412 27182 catalog_manager.cc:4841] TS ef604f29695544c095a058123660345a (127.20.125.130:45523): tablet 8623e355aa494e79a8e13554b24d857d (table test-table [id=aad70c506c344718934faefc10d4cc39]) successfully deleted
W20260809 09:32:49.159649 27184 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.20.125.129:41219 (error 108)
I20260809 09:32:49.159726 20982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu with pid 27508
I20260809 09:32:49.167152 27892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:49.166716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.129:41219 (local address 127.0.0.1:60100)
0809 09:32:49.166926 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0809 09:32:49.167009 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.129:41219: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114}
W20260809 09:32:49.167452 27184 catalog_manager.cc:4584] TS 6f03935c0ae54f6c8c48865f4b870ef8 (127.20.125.129:41219): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 8623e355aa494e79a8e13554b24d857d: Network error: Client connection negotiation failed: client connection to 127.20.125.129:41219: connect: Connection refused (error 111)
I20260809 09:32:49.179131 20982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu with pid 27691
I20260809 09:32:49.204449 20982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu with pid 27158
2026-08-09T09:32:49Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2764 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-09T09:32:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T09:32:49Z Disabled control of system clock
I20260809 09:32:49.244702 20982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
/tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786267941944493-20982-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786267941944493-20982-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786267941944493-20982-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786267941944493-20982-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.125.190:35329
--webserver_interface=127.20.125.190
--webserver_port=0
--builtin_ntp_servers=127.20.125.148:34243
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.125.190:35329
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260809 09:32:49.357905 27946 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 09:32:49.358155 27946 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 09:32:49.358191 27946 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 09:32:49.362635 27946 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 09:32:49.362691 27946 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 09:32:49.362707 27946 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 09:32:49.362723 27946 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 09:32:49.362738 27946 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 09:32:49.367333 27946 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.125.148:34243
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786267941944493-20982-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786267941944493-20982-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.125.190:35329
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.125.190:35329
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786267941944493-20982-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.125.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27946
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786267941944493-20982-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 08:38:55 UTC on ccycloud.quasar-njjleu.root.comops.site
build id 4575935
I20260809 09:32:49.368319 27946 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 09:32:49.369639 27946 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 09:32:49.378460 27946 server_base.cc:1034] running on GCE node
W20260809 09:32:49.378368 27953 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:32:49.378535 27955 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:32:49.378389 27957 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:32:49.379276 27946 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 09:32:49.380426 27946 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 09:32:49.381608 27946 hybrid_clock.cc:648] HybridClock initialized: now 1786267969381570 us; error 49 us; skew 500 ppm
I20260809 09:32:49.383893 27946 webserver.cc:460] Webserver started at http://127.20.125.190:37963/ using document root <none> and password file <none>
I20260809 09:32:49.384506 27946 fs_manager.cc:359] Metadata directory not provided
I20260809 09:32:49.384572 27946 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 09:32:49.384799 27946 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 09:32:49.387007 27946 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786267941944493-20982-0/minicluster-data/master-0/data/instance:
uuid: "d6e9b8159ab94bc3b57260a9c9d80ec6"
format_stamp: "Formatted at 2026-08-09 09:32:49 on dist-test-slave-7kzw"
server_key: "a8656d8243fc9ae2915c96fc071feaaf"
server_key_iv: "a14ec4ba751c5deb2fee7b4c5094ab7c"
server_key_version: "encryptionkey@0"
I20260809 09:32:49.387620 27946 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786267941944493-20982-0/minicluster-data/master-0/wal/instance:
uuid: "d6e9b8159ab94bc3b57260a9c9d80ec6"
format_stamp: "Formatted at 2026-08-09 09:32:49 on dist-test-slave-7kzw"
server_key: "a8656d8243fc9ae2915c96fc071feaaf"
server_key_iv: "a14ec4ba751c5deb2fee7b4c5094ab7c"
server_key_version: "encryptionkey@0"
I20260809 09:32:49.391650 27946 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.004s
I20260809 09:32:49.394284 27966 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 09:32:49.395655 27946 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 09:32:49.395859 27946 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786267941944493-20982-0/minicluster-data/master-0/data,/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786267941944493-20982-0/minicluster-data/master-0/wal
uuid: "d6e9b8159ab94bc3b57260a9c9d80ec6"
format_stamp: "Formatted at 2026-08-09 09:32:49 on dist-test-slave-7kzw"
server_key: "a8656d8243fc9ae2915c96fc071feaaf"
server_key_iv: "a14ec4ba751c5deb2fee7b4c5094ab7c"
server_key_version: "encryptionkey@0"
I20260809 09:32:49.395988 27946 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786267941944493-20982-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786267941944493-20982-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786267941944493-20982-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 09:32:49.400900 27946 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 09:32:49.401540 27946 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 09:32:49.401736 27946 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 09:32:49.409019 27946 rpc_server.cc:307] RPC server started. Bound to: 127.20.125.190:35329
I20260809 09:32:49.409034 28043 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.125.190:35329 every 8 connection(s)
I20260809 09:32:49.410000 27946 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786267941944493-20982-0/minicluster-data/master-0/data/info.pb
I20260809 09:32:49.410754 20982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu as pid 27946
I20260809 09:32:49.410885 20982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786267941944493-20982-0/minicluster-data/master-0/wal/instance
I20260809 09:32:49.411171 20982 external_mini_cluster.cc:1428] Setting key 824f47a869d6b0c8bb76bcd62d35c085
I20260809 09:32:49.413537 28044 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 09:32:49.417753 28049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:49.412699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48656 (local address 127.20.125.190:35329)
0809 09:32:49.414063 (+ 1364us) server_negotiation.cc:206] Beginning negotiation
0809 09:32:49.414079 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 09:32:49.414116 (+ 37us) server_negotiation.cc:407] Connection header received
0809 09:32:49.414856 (+ 740us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:32:49.414876 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:32:49.415258 (+ 382us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:32:49.415596 (+ 338us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:32:49.416487 (+ 891us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:32:49.416491 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:32:49.416503 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 09:32:49.416514 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:32:49.416570 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 09:32:49.416587 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:32:49.416737 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0809 09:32:49.416811 (+ 74us) server_negotiation.cc:299] Negotiation successful
0809 09:32:49.417063 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":998,"thread_start_us":798,"threads_started":1}
I20260809 09:32:49.420532 28044 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d6e9b8159ab94bc3b57260a9c9d80ec6: Bootstrap starting.
I20260809 09:32:49.423023 28044 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d6e9b8159ab94bc3b57260a9c9d80ec6: Neither blocks nor log segments found. Creating new log.
I20260809 09:32:49.423935 28044 log.cc:824] T 00000000000000000000000000000000 P d6e9b8159ab94bc3b57260a9c9d80ec6: Log is configured to *not* fsync() on all Append() calls
I20260809 09:32:49.426065 28044 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d6e9b8159ab94bc3b57260a9c9d80ec6: No bootstrap required, opened a new log
I20260809 09:32:49.430151 28044 raft_consensus.cc:348] T 00000000000000000000000000000000 P d6e9b8159ab94bc3b57260a9c9d80ec6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6e9b8159ab94bc3b57260a9c9d80ec6" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 35329 } }
I20260809 09:32:49.430513 28044 raft_consensus.cc:374] T 00000000000000000000000000000000 P d6e9b8159ab94bc3b57260a9c9d80ec6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 09:32:49.430598 28044 raft_consensus.cc:729] T 00000000000000000000000000000000 P d6e9b8159ab94bc3b57260a9c9d80ec6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6e9b8159ab94bc3b57260a9c9d80ec6, State: Initialized, Role: FOLLOWER
I20260809 09:32:49.430989 28044 consensus_queue.cc:260] T 00000000000000000000000000000000 P d6e9b8159ab94bc3b57260a9c9d80ec6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6e9b8159ab94bc3b57260a9c9d80ec6" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 35329 } }
I20260809 09:32:49.431116 28044 raft_consensus.cc:388] T 00000000000000000000000000000000 P d6e9b8159ab94bc3b57260a9c9d80ec6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 09:32:49.431159 28044 raft_consensus.cc:482] T 00000000000000000000000000000000 P d6e9b8159ab94bc3b57260a9c9d80ec6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 09:32:49.431228 28044 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d6e9b8159ab94bc3b57260a9c9d80ec6 [term 0 FOLLOWER]: Advancing to term 1
I20260809 09:32:49.432124 28044 raft_consensus.cc:504] T 00000000000000000000000000000000 P d6e9b8159ab94bc3b57260a9c9d80ec6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6e9b8159ab94bc3b57260a9c9d80ec6" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 35329 } }
I20260809 09:32:49.432435 28044 leader_election.cc:302] T 00000000000000000000000000000000 P d6e9b8159ab94bc3b57260a9c9d80ec6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d6e9b8159ab94bc3b57260a9c9d80ec6; no voters:
I20260809 09:32:49.432714 28044 leader_election.cc:288] T 00000000000000000000000000000000 P d6e9b8159ab94bc3b57260a9c9d80ec6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 09:32:49.432860 28053 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d6e9b8159ab94bc3b57260a9c9d80ec6 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 09:32:49.433079 28053 raft_consensus.cc:686] T 00000000000000000000000000000000 P d6e9b8159ab94bc3b57260a9c9d80ec6 [term 1 LEADER]: Becoming Leader. State: Replica: d6e9b8159ab94bc3b57260a9c9d80ec6, State: Running, Role: LEADER
I20260809 09:32:49.433385 28053 consensus_queue.cc:237] T 00000000000000000000000000000000 P d6e9b8159ab94bc3b57260a9c9d80ec6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6e9b8159ab94bc3b57260a9c9d80ec6" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 35329 } }
I20260809 09:32:49.434051 28044 sys_catalog.cc:564] T 00000000000000000000000000000000 P d6e9b8159ab94bc3b57260a9c9d80ec6 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 09:32:49.435195 28056 sys_catalog.cc:455] T 00000000000000000000000000000000 P d6e9b8159ab94bc3b57260a9c9d80ec6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d6e9b8159ab94bc3b57260a9c9d80ec6. Latest consensus state: current_term: 1 leader_uuid: "d6e9b8159ab94bc3b57260a9c9d80ec6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6e9b8159ab94bc3b57260a9c9d80ec6" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 35329 } } }
I20260809 09:32:49.435359 28056 sys_catalog.cc:458] T 00000000000000000000000000000000 P d6e9b8159ab94bc3b57260a9c9d80ec6 [sys.catalog]: This master's current role is: LEADER
I20260809 09:32:49.435678 28054 sys_catalog.cc:455] T 00000000000000000000000000000000 P d6e9b8159ab94bc3b57260a9c9d80ec6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d6e9b8159ab94bc3b57260a9c9d80ec6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6e9b8159ab94bc3b57260a9c9d80ec6" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 35329 } } }
I20260809 09:32:49.435779 28070 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 09:32:49.435789 28054 sys_catalog.cc:458] T 00000000000000000000000000000000 P d6e9b8159ab94bc3b57260a9c9d80ec6 [sys.catalog]: This master's current role is: LEADER
I20260809 09:32:49.439139 28070 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 09:32:49.445346 28070 catalog_manager.cc:1347] Generated new cluster ID: 686564f7ba94419aa58741ea0114476b
I20260809 09:32:49.445432 28070 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 09:32:49.466737 28070 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 09:32:49.467073 28070 catalog_manager.cc:1504] Loading token signing keys...
I20260809 09:32:49.473995 28070 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d6e9b8159ab94bc3b57260a9c9d80ec6: Generated new TSK 0
I20260809 09:32:49.474920 28070 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 09:32:49.490789 20982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
/tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786267941944493-20982-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786267941944493-20982-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786267941944493-20982-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786267941944493-20982-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.125.129:0
--local_ip_for_outbound_sockets=127.20.125.129
--webserver_interface=127.20.125.129
--webserver_port=0
--tserver_master_addrs=127.20.125.190:35329
--builtin_ntp_servers=127.20.125.148:34243
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260809 09:32:49.613363 28084 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 09:32:49.613654 28084 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 09:32:49.613696 28084 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 09:32:49.617936 28084 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 09:32:49.617991 28084 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 09:32:49.618072 28084 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.125.129
I20260809 09:32:49.622961 28084 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.125.148:34243
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786267941944493-20982-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786267941944493-20982-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.125.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786267941944493-20982-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.125.129
--webserver_port=0
--tserver_master_addrs=127.20.125.190:35329
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28084
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.125.129
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786267941944493-20982-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 08:38:55 UTC on ccycloud.quasar-njjleu.root.comops.site
build id 4575935
I20260809 09:32:49.624073 28084 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 09:32:49.625492 28084 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 09:32:49.635242 28096 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:32:49.635396 28092 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:32:49.635367 28094 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:32:49.635495 28084 server_base.cc:1034] running on GCE node
I20260809 09:32:49.636032 28084 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 09:32:49.636672 28084 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 09:32:49.637902 28084 hybrid_clock.cc:648] HybridClock initialized: now 1786267969637889 us; error 27 us; skew 500 ppm
I20260809 09:32:49.639889 28084 webserver.cc:460] Webserver started at http://127.20.125.129:39751/ using document root <none> and password file <none>
I20260809 09:32:49.640578 28084 fs_manager.cc:359] Metadata directory not provided
I20260809 09:32:49.640658 28084 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 09:32:49.640903 28084 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 09:32:49.643100 28084 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786267941944493-20982-0/minicluster-data/ts-0/data/instance:
uuid: "ae0b8ab8e3e34b6ca52a2c2904626af0"
format_stamp: "Formatted at 2026-08-09 09:32:49 on dist-test-slave-7kzw"
server_key: "ad4b18093aac927f2a229e46504544af"
server_key_iv: "16fc00d3d01121a0d6d717fd31ef5455"
server_key_version: "encryptionkey@0"
I20260809 09:32:49.643687 28084 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786267941944493-20982-0/minicluster-data/ts-0/wal/instance:
uuid: "ae0b8ab8e3e34b6ca52a2c2904626af0"
format_stamp: "Formatted at 2026-08-09 09:32:49 on dist-test-slave-7kzw"
server_key: "ad4b18093aac927f2a229e46504544af"
server_key_iv: "16fc00d3d01121a0d6d717fd31ef5455"
server_key_version: "encryptionkey@0"
I20260809 09:32:49.647465 28084 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260809 09:32:49.649999 28106 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 09:32:49.651252 28084 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260809 09:32:49.651384 28084 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786267941944493-20982-0/minicluster-data/ts-0/data,/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786267941944493-20982-0/minicluster-data/ts-0/wal
uuid: "ae0b8ab8e3e34b6ca52a2c2904626af0"
format_stamp: "Formatted at 2026-08-09 09:32:49 on dist-test-slave-7kzw"
server_key: "ad4b18093aac927f2a229e46504544af"
server_key_iv: "16fc00d3d01121a0d6d717fd31ef5455"
server_key_version: "encryptionkey@0"
I20260809 09:32:49.651499 28084 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786267941944493-20982-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786267941944493-20982-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786267941944493-20982-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 09:32:49.656252 28084 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 09:32:49.656853 28084 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 09:32:49.657032 28084 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 09:32:49.657570 28084 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 09:32:49.658540 28084 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 09:32:49.658607 28084 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 09:32:49.658667 28084 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 09:32:49.658715 28084 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 09:32:49.669612 28084 rpc_server.cc:307] RPC server started. Bound to: 127.20.125.129:38647
I20260809 09:32:49.669685 28280 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.125.129:38647 every 8 connection(s)
I20260809 09:32:49.670624 28084 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786267941944493-20982-0/minicluster-data/ts-0/data/info.pb
I20260809 09:32:49.676754 28049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:49.672444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.129:55505 (local address 127.20.125.190:35329)
0809 09:32:49.672578 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0809 09:32:49.672582 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 09:32:49.673632 (+ 1050us) server_negotiation.cc:407] Connection header received
0809 09:32:49.675182 (+ 1550us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:32:49.675186 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:32:49.675233 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:32:49.675314 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:32:49.676314 (+ 1000us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:32:49.676317 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:32:49.676318 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 09:32:49.676328 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:32:49.676345 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 09:32:49.676352 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:32:49.676430 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0809 09:32:49.676583 (+ 153us) server_negotiation.cc:299] Negotiation successful
0809 09:32:49.676627 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260809 09:32:49.677533 28284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:49.672781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:35329 (local address 127.20.125.129:55505)
0809 09:32:49.673498 (+ 717us) negotiation.cc:107] Waiting for socket to connect
0809 09:32:49.673538 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 09:32:49.674972 (+ 1434us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:32:49.675441 (+ 469us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:32:49.675448 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:32:49.675533 (+ 85us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:32:49.676066 (+ 533us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 09:32:49.676068 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:32:49.676107 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 09:32:49.676188 (+ 81us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 09:32:49.676425 (+ 237us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 09:32:49.676432 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 09:32:49.676436 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 09:32:49.676565 (+ 129us) client_negotiation.cc:240] Negotiation successful
0809 09:32:49.676751 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":294,"thread_start_us":127,"threads_started":1}
I20260809 09:32:49.678974 20982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu as pid 28084
I20260809 09:32:49.679102 20982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786267941944493-20982-0/minicluster-data/ts-0/wal/instance
I20260809 09:32:49.679102 28281 heartbeater.cc:347] Connected to a master server at 127.20.125.190:35329
I20260809 09:32:49.679320 28281 heartbeater.cc:464] Registering TS with master...
I20260809 09:32:49.679409 20982 external_mini_cluster.cc:1428] Setting key 876132231086b8550008b46c7a6f6e85
I20260809 09:32:49.679765 28281 heartbeater.cc:511] Master 127.20.125.190:35329 requested a full tablet report, sending...
I20260809 09:32:49.681566 20982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
/tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786267941944493-20982-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786267941944493-20982-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786267941944493-20982-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786267941944493-20982-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.125.130:0
--local_ip_for_outbound_sockets=127.20.125.130
--webserver_interface=127.20.125.130
--webserver_port=0
--tserver_master_addrs=127.20.125.190:35329
--builtin_ntp_servers=127.20.125.148:34243
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260809 09:32:49.681555 27988 ts_manager.cc:194] Registered new tserver with Master: ae0b8ab8e3e34b6ca52a2c2904626af0 (127.20.125.129:38647)
W20260809 09:32:49.798010 28285 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 09:32:49.798295 28285 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 09:32:49.798369 28285 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 09:32:49.802688 28285 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 09:32:49.802752 28285 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 09:32:49.802809 28285 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.125.130
I20260809 09:32:49.807581 28285 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.125.148:34243
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786267941944493-20982-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786267941944493-20982-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.125.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786267941944493-20982-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.125.130
--webserver_port=0
--tserver_master_addrs=127.20.125.190:35329
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28285
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.125.130
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786267941944493-20982-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 08:38:55 UTC on ccycloud.quasar-njjleu.root.comops.site
build id 4575935
I20260809 09:32:49.808739 28285 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 09:32:49.810161 28285 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 09:32:49.819984 28291 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:32:49.820029 28296 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:32:49.820178 28285 server_base.cc:1034] running on GCE node
W20260809 09:32:49.820029 28294 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:32:49.820611 28285 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 09:32:49.821228 28285 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 09:32:49.822434 28285 hybrid_clock.cc:648] HybridClock initialized: now 1786267969822404 us; error 46 us; skew 500 ppm
I20260809 09:32:49.824254 28285 webserver.cc:460] Webserver started at http://127.20.125.130:41415/ using document root <none> and password file <none>
I20260809 09:32:49.824836 28285 fs_manager.cc:359] Metadata directory not provided
I20260809 09:32:49.824929 28285 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 09:32:49.825160 28285 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 09:32:49.827267 28285 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786267941944493-20982-0/minicluster-data/ts-1/data/instance:
uuid: "b90e1d0df8134a56981dd860a74ddaf4"
format_stamp: "Formatted at 2026-08-09 09:32:49 on dist-test-slave-7kzw"
server_key: "971c7aada71045911e846949bacd87e9"
server_key_iv: "4c49ab28a1236bfafeb248bd93ffbde3"
server_key_version: "encryptionkey@0"
I20260809 09:32:49.827831 28285 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786267941944493-20982-0/minicluster-data/ts-1/wal/instance:
uuid: "b90e1d0df8134a56981dd860a74ddaf4"
format_stamp: "Formatted at 2026-08-09 09:32:49 on dist-test-slave-7kzw"
server_key: "971c7aada71045911e846949bacd87e9"
server_key_iv: "4c49ab28a1236bfafeb248bd93ffbde3"
server_key_version: "encryptionkey@0"
I20260809 09:32:49.831506 28285 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260809 09:32:49.834120 28303 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 09:32:49.835497 28285 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 09:32:49.835629 28285 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786267941944493-20982-0/minicluster-data/ts-1/data,/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786267941944493-20982-0/minicluster-data/ts-1/wal
uuid: "b90e1d0df8134a56981dd860a74ddaf4"
format_stamp: "Formatted at 2026-08-09 09:32:49 on dist-test-slave-7kzw"
server_key: "971c7aada71045911e846949bacd87e9"
server_key_iv: "4c49ab28a1236bfafeb248bd93ffbde3"
server_key_version: "encryptionkey@0"
I20260809 09:32:49.835742 28285 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786267941944493-20982-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786267941944493-20982-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786267941944493-20982-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 09:32:49.840373 28285 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 09:32:49.840956 28285 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 09:32:49.841132 28285 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 09:32:49.841684 28285 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 09:32:49.842638 28285 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 09:32:49.842703 28285 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 09:32:49.842770 28285 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 09:32:49.842818 28285 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 09:32:49.851307 28285 rpc_server.cc:307] RPC server started. Bound to: 127.20.125.130:38155
I20260809 09:32:49.851397 28469 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.125.130:38155 every 8 connection(s)
I20260809 09:32:49.852210 28285 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786267941944493-20982-0/minicluster-data/ts-1/data/info.pb
I20260809 09:32:49.858219 28049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:49.854065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.130:57729 (local address 127.20.125.190:35329)
0809 09:32:49.854228 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0809 09:32:49.854233 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 09:32:49.855307 (+ 1074us) server_negotiation.cc:407] Connection header received
0809 09:32:49.856612 (+ 1305us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:32:49.856615 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:32:49.856653 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:32:49.856735 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:32:49.857836 (+ 1101us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:32:49.857838 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:32:49.857840 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 09:32:49.857852 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:32:49.857871 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 09:32:49.857879 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:32:49.857957 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0809 09:32:49.858055 (+ 98us) server_negotiation.cc:299] Negotiation successful
0809 09:32:49.858091 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260809 09:32:49.859074 28472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:49.854427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:35329 (local address 127.20.125.130:57729)
0809 09:32:49.855149 (+ 722us) negotiation.cc:107] Waiting for socket to connect
0809 09:32:49.855190 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 09:32:49.856419 (+ 1229us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:32:49.856864 (+ 445us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:32:49.856870 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:32:49.856951 (+ 81us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:32:49.857540 (+ 589us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 09:32:49.857541 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:32:49.857582 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 09:32:49.857656 (+ 74us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 09:32:49.857948 (+ 292us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 09:32:49.857952 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 09:32:49.857955 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 09:32:49.858074 (+ 119us) client_negotiation.cc:240] Negotiation successful
0809 09:32:49.858257 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":330,"thread_start_us":118,"threads_started":1}
I20260809 09:32:49.859691 20982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu as pid 28285
I20260809 09:32:49.859820 20982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786267941944493-20982-0/minicluster-data/ts-1/wal/instance
I20260809 09:32:49.860158 20982 external_mini_cluster.cc:1428] Setting key bd3650878d3a6fbb34ae436390e7adc3
I20260809 09:32:49.860415 28470 heartbeater.cc:347] Connected to a master server at 127.20.125.190:35329
I20260809 09:32:49.860690 28470 heartbeater.cc:464] Registering TS with master...
I20260809 09:32:49.861104 28470 heartbeater.cc:511] Master 127.20.125.190:35329 requested a full tablet report, sending...
I20260809 09:32:49.862167 27988 ts_manager.cc:194] Registered new tserver with Master: b90e1d0df8134a56981dd860a74ddaf4 (127.20.125.130:38155)
I20260809 09:32:49.862397 20982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
/tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786267941944493-20982-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786267941944493-20982-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786267941944493-20982-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786267941944493-20982-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.125.131:0
--local_ip_for_outbound_sockets=127.20.125.131
--webserver_interface=127.20.125.131
--webserver_port=0
--tserver_master_addrs=127.20.125.190:35329
--builtin_ntp_servers=127.20.125.148:34243
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260809 09:32:49.979429 28473 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 09:32:49.979733 28473 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 09:32:49.979837 28473 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 09:32:49.983870 28473 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 09:32:49.983945 28473 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 09:32:49.984028 28473 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.125.131
I20260809 09:32:49.988858 28473 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.125.148:34243
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786267941944493-20982-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786267941944493-20982-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.125.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786267941944493-20982-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.125.131
--webserver_port=0
--tserver_master_addrs=127.20.125.190:35329
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28473
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.125.131
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786267941944493-20982-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 08:38:55 UTC on ccycloud.quasar-njjleu.root.comops.site
build id 4575935
I20260809 09:32:49.989940 28473 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 09:32:49.991380 28473 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 09:32:50.001435 28480 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:32:50.001600 28473 server_base.cc:1034] running on GCE node
W20260809 09:32:50.001441 28484 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:32:50.001447 28479 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:32:50.002198 28473 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 09:32:50.002890 28473 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 09:32:50.004148 28473 hybrid_clock.cc:648] HybridClock initialized: now 1786267970004109 us; error 45 us; skew 500 ppm
I20260809 09:32:50.006279 28473 webserver.cc:460] Webserver started at http://127.20.125.131:46585/ using document root <none> and password file <none>
I20260809 09:32:50.006969 28473 fs_manager.cc:359] Metadata directory not provided
I20260809 09:32:50.007078 28473 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 09:32:50.007333 28473 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 09:32:50.009447 28473 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786267941944493-20982-0/minicluster-data/ts-2/data/instance:
uuid: "f84301c7610748da896c51037f844983"
format_stamp: "Formatted at 2026-08-09 09:32:50 on dist-test-slave-7kzw"
server_key: "b8899430ebc2fc9e42dd8ca9af01bfed"
server_key_iv: "56c08e39d99655a10fc9f68e0fa7ab4b"
server_key_version: "encryptionkey@0"
I20260809 09:32:50.010031 28473 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786267941944493-20982-0/minicluster-data/ts-2/wal/instance:
uuid: "f84301c7610748da896c51037f844983"
format_stamp: "Formatted at 2026-08-09 09:32:50 on dist-test-slave-7kzw"
server_key: "b8899430ebc2fc9e42dd8ca9af01bfed"
server_key_iv: "56c08e39d99655a10fc9f68e0fa7ab4b"
server_key_version: "encryptionkey@0"
I20260809 09:32:50.013830 28473 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260809 09:32:50.016510 28492 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 09:32:50.017717 28473 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260809 09:32:50.017864 28473 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786267941944493-20982-0/minicluster-data/ts-2/data,/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786267941944493-20982-0/minicluster-data/ts-2/wal
uuid: "f84301c7610748da896c51037f844983"
format_stamp: "Formatted at 2026-08-09 09:32:50 on dist-test-slave-7kzw"
server_key: "b8899430ebc2fc9e42dd8ca9af01bfed"
server_key_iv: "56c08e39d99655a10fc9f68e0fa7ab4b"
server_key_version: "encryptionkey@0"
I20260809 09:32:50.018003 28473 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786267941944493-20982-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786267941944493-20982-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786267941944493-20982-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 09:32:50.022773 28473 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 09:32:50.023396 28473 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 09:32:50.023579 28473 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 09:32:50.024138 28473 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 09:32:50.025101 28473 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 09:32:50.025169 28473 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 09:32:50.025234 28473 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 09:32:50.025280 28473 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 09:32:50.033984 28473 rpc_server.cc:307] RPC server started. Bound to: 127.20.125.131:39537
I20260809 09:32:50.034056 28663 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.125.131:39537 every 8 connection(s)
I20260809 09:32:50.034958 28473 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786267941944493-20982-0/minicluster-data/ts-2/data/info.pb
I20260809 09:32:50.039630 20982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu as pid 28473
I20260809 09:32:50.039784 20982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786267941944493-20982-0/minicluster-data/ts-2/wal/instance
I20260809 09:32:50.040068 20982 external_mini_cluster.cc:1428] Setting key 92a3be1ac1e8d6b468f7a683852b95c7
I20260809 09:32:50.041895 28049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:50.036832 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.131:33903 (local address 127.20.125.190:35329)
0809 09:32:50.036994 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0809 09:32:50.036999 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 09:32:50.038124 (+ 1125us) server_negotiation.cc:407] Connection header received
0809 09:32:50.039667 (+ 1543us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:32:50.039673 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:32:50.039735 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:32:50.039871 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:32:50.041357 (+ 1486us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:32:50.041361 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:32:50.041365 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 09:32:50.041378 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:32:50.041409 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 09:32:50.041417 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:32:50.041514 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0809 09:32:50.041673 (+ 159us) server_negotiation.cc:299] Negotiation successful
0809 09:32:50.041709 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260809 09:32:50.042951 28667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:50.037315 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:35329 (local address 127.20.125.131:33903)
0809 09:32:50.037977 (+ 662us) negotiation.cc:107] Waiting for socket to connect
0809 09:32:50.038018 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 09:32:50.039313 (+ 1295us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:32:50.040057 (+ 744us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:32:50.040065 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:32:50.040194 (+ 129us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:32:50.040977 (+ 783us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 09:32:50.040981 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:32:50.041064 (+ 83us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 09:32:50.041193 (+ 129us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 09:32:50.041512 (+ 319us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 09:32:50.041523 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 09:32:50.041532 (+ 9us) client_negotiation.cc:769] Sending connection context
0809 09:32:50.041728 (+ 196us) client_negotiation.cc:240] Negotiation successful
0809 09:32:50.041990 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":277,"thread_start_us":134,"threads_started":1}
I20260809 09:32:50.044112 28664 heartbeater.cc:347] Connected to a master server at 127.20.125.190:35329
I20260809 09:32:50.044363 28664 heartbeater.cc:464] Registering TS with master...
I20260809 09:32:50.044755 28664 heartbeater.cc:511] Master 127.20.125.190:35329 requested a full tablet report, sending...
I20260809 09:32:50.045800 27988 ts_manager.cc:194] Registered new tserver with Master: f84301c7610748da896c51037f844983 (127.20.125.131:39537)
I20260809 09:32:50.054519 20982 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 09:32:50.058915 28049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:50.056896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48662 (local address 127.20.125.190:35329)
0809 09:32:50.057033 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0809 09:32:50.057038 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 09:32:50.057254 (+ 216us) server_negotiation.cc:407] Connection header received
0809 09:32:50.057397 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:32:50.057400 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:32:50.057440 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:32:50.057518 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:32:50.058337 (+ 819us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:32:50.058340 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:32:50.058343 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 09:32:50.058355 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:32:50.058376 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 09:32:50.058384 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:32:50.058657 (+ 273us) server_negotiation.cc:1035] Waiting for connection context
0809 09:32:50.058701 (+ 44us) server_negotiation.cc:299] Negotiation successful
0809 09:32:50.058752 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260809 09:32:50.061916 27988 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48662:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 09:32:50.065249 27988 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 09:32:50.077775 28685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:50.074425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.131:39537 (local address 127.0.0.1:43388)
0809 09:32:50.074792 (+ 367us) negotiation.cc:107] Waiting for socket to connect
0809 09:32:50.074808 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 09:32:50.074902 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:32:50.076308 (+ 1406us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:32:50.076315 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:32:50.076332 (+ 17us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:32:50.076846 (+ 514us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 09:32:50.076848 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:32:50.076860 (+ 12us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 09:32:50.076889 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 09:32:50.077447 (+ 558us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 09:32:50.077451 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 09:32:50.077455 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 09:32:50.077548 (+ 93us) client_negotiation.cc:240] Negotiation successful
0809 09:32:50.077586 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":302,"thread_start_us":135,"threads_started":1}
I20260809 09:32:50.077808 28689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:50.075334 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43388 (local address 127.20.125.131:39537)
0809 09:32:50.075827 (+ 493us) server_negotiation.cc:206] Beginning negotiation
0809 09:32:50.075841 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 09:32:50.075868 (+ 27us) server_negotiation.cc:407] Connection header received
0809 09:32:50.075929 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:32:50.075934 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:32:50.076121 (+ 187us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:32:50.076544 (+ 423us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:32:50.077211 (+ 667us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:32:50.077217 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:32:50.077226 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 09:32:50.077240 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:32:50.077283 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 09:32:50.077311 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:32:50.077462 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0809 09:32:50.077560 (+ 98us) server_negotiation.cc:299] Negotiation successful
0809 09:32:50.077636 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":391,"thread_start_us":161,"threads_started":1}
I20260809 09:32:50.078384 28682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:50.074347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44408 (local address 127.20.125.130:38155)
0809 09:32:50.075293 (+ 946us) server_negotiation.cc:206] Beginning negotiation
0809 09:32:50.075308 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 09:32:50.075354 (+ 46us) server_negotiation.cc:407] Connection header received
0809 09:32:50.075431 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:32:50.075437 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:32:50.075657 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:32:50.075806 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:32:50.076988 (+ 1182us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:32:50.076995 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:32:50.077008 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 09:32:50.077021 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:32:50.077071 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 09:32:50.077106 (+ 35us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:32:50.077224 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0809 09:32:50.078127 (+ 903us) server_negotiation.cc:299] Negotiation successful
0809 09:32:50.078207 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":738,"thread_start_us":174,"threads_started":1}
I20260809 09:32:50.077682 28681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:50.074158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.130:38155 (local address 127.0.0.1:44408)
0809 09:32:50.074642 (+ 484us) negotiation.cc:107] Waiting for socket to connect
0809 09:32:50.074701 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0809 09:32:50.074806 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:32:50.075944 (+ 1138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:32:50.075951 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:32:50.076033 (+ 82us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:32:50.076705 (+ 672us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 09:32:50.076708 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:32:50.076730 (+ 22us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 09:32:50.076806 (+ 76us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 09:32:50.077275 (+ 469us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 09:32:50.077285 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 09:32:50.077292 (+ 7us) client_negotiation.cc:769] Sending connection context
0809 09:32:50.077415 (+ 123us) client_negotiation.cc:240] Negotiation successful
0809 09:32:50.077494 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":118,"threads_started":1}
I20260809 09:32:50.079115 28690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:50.075413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52334 (local address 127.20.125.129:38647)
0809 09:32:50.075830 (+ 417us) server_negotiation.cc:206] Beginning negotiation
0809 09:32:50.075842 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 09:32:50.076093 (+ 251us) server_negotiation.cc:407] Connection header received
0809 09:32:50.076170 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:32:50.076178 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:32:50.076929 (+ 751us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:32:50.077060 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:32:50.078227 (+ 1167us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:32:50.078232 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:32:50.078241 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 09:32:50.078250 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:32:50.078286 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 09:32:50.078308 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:32:50.078410 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0809 09:32:50.078889 (+ 479us) server_negotiation.cc:299] Negotiation successful
0809 09:32:50.078965 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":329,"thread_start_us":97,"threads_started":1}
I20260809 09:32:50.079428 28688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:50.075031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.129:38647 (local address 127.0.0.1:52334)
0809 09:32:50.075900 (+ 869us) negotiation.cc:107] Waiting for socket to connect
0809 09:32:50.075914 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 09:32:50.075992 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:32:50.077040 (+ 1048us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:32:50.077042 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:32:50.077056 (+ 14us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:32:50.077741 (+ 685us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 09:32:50.077743 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:32:50.077752 (+ 9us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 09:32:50.077776 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 09:32:50.078452 (+ 676us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 09:32:50.078455 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 09:32:50.078458 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 09:32:50.078549 (+ 91us) client_negotiation.cc:240] Negotiation successful
0809 09:32:50.079292 (+ 743us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":809,"thread_start_us":128,"threads_started":1}
I20260809 09:32:50.081350 28376 tablet_service.cc:1467] Processing CreateTablet for tablet 97e424f0af764ef1a1a0b15854ec6553 (DEFAULT_TABLE table=test-table [id=5f7714d3366c40369068e5aa636a2c5b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 09:32:50.081343 28566 tablet_service.cc:1467] Processing CreateTablet for tablet 97e424f0af764ef1a1a0b15854ec6553 (DEFAULT_TABLE table=test-table [id=5f7714d3366c40369068e5aa636a2c5b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 09:32:50.081691 28156 tablet_service.cc:1467] Processing CreateTablet for tablet 97e424f0af764ef1a1a0b15854ec6553 (DEFAULT_TABLE table=test-table [id=5f7714d3366c40369068e5aa636a2c5b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 09:32:50.082613 28376 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 97e424f0af764ef1a1a0b15854ec6553. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 09:32:50.082621 28566 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 97e424f0af764ef1a1a0b15854ec6553. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 09:32:50.082777 28156 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 97e424f0af764ef1a1a0b15854ec6553. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 09:32:50.089037 28692 tablet_bootstrap.cc:492] T 97e424f0af764ef1a1a0b15854ec6553 P ae0b8ab8e3e34b6ca52a2c2904626af0: Bootstrap starting.
I20260809 09:32:50.090490 28694 tablet_bootstrap.cc:492] T 97e424f0af764ef1a1a0b15854ec6553 P f84301c7610748da896c51037f844983: Bootstrap starting.
I20260809 09:32:50.091188 28693 tablet_bootstrap.cc:492] T 97e424f0af764ef1a1a0b15854ec6553 P b90e1d0df8134a56981dd860a74ddaf4: Bootstrap starting.
I20260809 09:32:50.092327 28692 tablet_bootstrap.cc:654] T 97e424f0af764ef1a1a0b15854ec6553 P ae0b8ab8e3e34b6ca52a2c2904626af0: Neither blocks nor log segments found. Creating new log.
I20260809 09:32:50.092677 28694 tablet_bootstrap.cc:654] T 97e424f0af764ef1a1a0b15854ec6553 P f84301c7610748da896c51037f844983: Neither blocks nor log segments found. Creating new log.
I20260809 09:32:50.093178 28692 log.cc:824] T 97e424f0af764ef1a1a0b15854ec6553 P ae0b8ab8e3e34b6ca52a2c2904626af0: Log is configured to *not* fsync() on all Append() calls
I20260809 09:32:50.093206 28693 tablet_bootstrap.cc:654] T 97e424f0af764ef1a1a0b15854ec6553 P b90e1d0df8134a56981dd860a74ddaf4: Neither blocks nor log segments found. Creating new log.
I20260809 09:32:50.093537 28694 log.cc:824] T 97e424f0af764ef1a1a0b15854ec6553 P f84301c7610748da896c51037f844983: Log is configured to *not* fsync() on all Append() calls
I20260809 09:32:50.094070 28693 log.cc:824] T 97e424f0af764ef1a1a0b15854ec6553 P b90e1d0df8134a56981dd860a74ddaf4: Log is configured to *not* fsync() on all Append() calls
I20260809 09:32:50.095471 28692 tablet_bootstrap.cc:492] T 97e424f0af764ef1a1a0b15854ec6553 P ae0b8ab8e3e34b6ca52a2c2904626af0: No bootstrap required, opened a new log
I20260809 09:32:50.095548 28694 tablet_bootstrap.cc:492] T 97e424f0af764ef1a1a0b15854ec6553 P f84301c7610748da896c51037f844983: No bootstrap required, opened a new log
I20260809 09:32:50.095686 28692 ts_tablet_manager.cc:1397] T 97e424f0af764ef1a1a0b15854ec6553 P ae0b8ab8e3e34b6ca52a2c2904626af0: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260809 09:32:50.095772 28694 ts_tablet_manager.cc:1397] T 97e424f0af764ef1a1a0b15854ec6553 P f84301c7610748da896c51037f844983: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260809 09:32:50.096553 28693 tablet_bootstrap.cc:492] T 97e424f0af764ef1a1a0b15854ec6553 P b90e1d0df8134a56981dd860a74ddaf4: No bootstrap required, opened a new log
I20260809 09:32:50.096741 28693 ts_tablet_manager.cc:1397] T 97e424f0af764ef1a1a0b15854ec6553 P b90e1d0df8134a56981dd860a74ddaf4: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260809 09:32:50.099776 28694 raft_consensus.cc:348] T 97e424f0af764ef1a1a0b15854ec6553 P f84301c7610748da896c51037f844983 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b90e1d0df8134a56981dd860a74ddaf4" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 38155 } } peers { permanent_uuid: "ae0b8ab8e3e34b6ca52a2c2904626af0" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 38647 } } peers { permanent_uuid: "f84301c7610748da896c51037f844983" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 39537 } }
I20260809 09:32:50.100098 28694 raft_consensus.cc:374] T 97e424f0af764ef1a1a0b15854ec6553 P f84301c7610748da896c51037f844983 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 09:32:50.100171 28694 raft_consensus.cc:729] T 97e424f0af764ef1a1a0b15854ec6553 P f84301c7610748da896c51037f844983 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f84301c7610748da896c51037f844983, State: Initialized, Role: FOLLOWER
I20260809 09:32:50.100374 28692 raft_consensus.cc:348] T 97e424f0af764ef1a1a0b15854ec6553 P ae0b8ab8e3e34b6ca52a2c2904626af0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b90e1d0df8134a56981dd860a74ddaf4" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 38155 } } peers { permanent_uuid: "ae0b8ab8e3e34b6ca52a2c2904626af0" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 38647 } } peers { permanent_uuid: "f84301c7610748da896c51037f844983" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 39537 } }
I20260809 09:32:50.100633 28692 raft_consensus.cc:374] T 97e424f0af764ef1a1a0b15854ec6553 P ae0b8ab8e3e34b6ca52a2c2904626af0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 09:32:50.100695 28692 raft_consensus.cc:729] T 97e424f0af764ef1a1a0b15854ec6553 P ae0b8ab8e3e34b6ca52a2c2904626af0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ae0b8ab8e3e34b6ca52a2c2904626af0, State: Initialized, Role: FOLLOWER
I20260809 09:32:50.100765 28694 consensus_queue.cc:260] T 97e424f0af764ef1a1a0b15854ec6553 P f84301c7610748da896c51037f844983 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b90e1d0df8134a56981dd860a74ddaf4" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 38155 } } peers { permanent_uuid: "ae0b8ab8e3e34b6ca52a2c2904626af0" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 38647 } } peers { permanent_uuid: "f84301c7610748da896c51037f844983" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 39537 } }
I20260809 09:32:50.101145 28692 consensus_queue.cc:260] T 97e424f0af764ef1a1a0b15854ec6553 P ae0b8ab8e3e34b6ca52a2c2904626af0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b90e1d0df8134a56981dd860a74ddaf4" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 38155 } } peers { permanent_uuid: "ae0b8ab8e3e34b6ca52a2c2904626af0" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 38647 } } peers { permanent_uuid: "f84301c7610748da896c51037f844983" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 39537 } }
I20260809 09:32:50.101383 28693 raft_consensus.cc:348] T 97e424f0af764ef1a1a0b15854ec6553 P b90e1d0df8134a56981dd860a74ddaf4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b90e1d0df8134a56981dd860a74ddaf4" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 38155 } } peers { permanent_uuid: "ae0b8ab8e3e34b6ca52a2c2904626af0" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 38647 } } peers { permanent_uuid: "f84301c7610748da896c51037f844983" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 39537 } }
I20260809 09:32:50.101639 28693 raft_consensus.cc:374] T 97e424f0af764ef1a1a0b15854ec6553 P b90e1d0df8134a56981dd860a74ddaf4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 09:32:50.101578 28664 heartbeater.cc:503] Master 127.20.125.190:35329 was elected leader, sending a full tablet report...
I20260809 09:32:50.101699 28693 raft_consensus.cc:729] T 97e424f0af764ef1a1a0b15854ec6553 P b90e1d0df8134a56981dd860a74ddaf4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b90e1d0df8134a56981dd860a74ddaf4, State: Initialized, Role: FOLLOWER
I20260809 09:32:50.101953 28694 ts_tablet_manager.cc:1428] T 97e424f0af764ef1a1a0b15854ec6553 P f84301c7610748da896c51037f844983: Time spent starting tablet: real 0.006s user 0.003s sys 0.004s
I20260809 09:32:50.102133 28692 ts_tablet_manager.cc:1428] T 97e424f0af764ef1a1a0b15854ec6553 P ae0b8ab8e3e34b6ca52a2c2904626af0: Time spent starting tablet: real 0.006s user 0.001s sys 0.006s
I20260809 09:32:50.102108 28693 consensus_queue.cc:260] T 97e424f0af764ef1a1a0b15854ec6553 P b90e1d0df8134a56981dd860a74ddaf4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b90e1d0df8134a56981dd860a74ddaf4" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 38155 } } peers { permanent_uuid: "ae0b8ab8e3e34b6ca52a2c2904626af0" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 38647 } } peers { permanent_uuid: "f84301c7610748da896c51037f844983" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 39537 } }
I20260809 09:32:50.102439 28281 heartbeater.cc:503] Master 127.20.125.190:35329 was elected leader, sending a full tablet report...
I20260809 09:32:50.104599 28470 heartbeater.cc:503] Master 127.20.125.190:35329 was elected leader, sending a full tablet report...
I20260809 09:32:50.104954 28693 ts_tablet_manager.cc:1428] T 97e424f0af764ef1a1a0b15854ec6553 P b90e1d0df8134a56981dd860a74ddaf4: Time spent starting tablet: real 0.008s user 0.006s sys 0.000s
I20260809 09:32:50.142584 28701 raft_consensus.cc:482] T 97e424f0af764ef1a1a0b15854ec6553 P b90e1d0df8134a56981dd860a74ddaf4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 09:32:50.142871 28701 raft_consensus.cc:504] T 97e424f0af764ef1a1a0b15854ec6553 P b90e1d0df8134a56981dd860a74ddaf4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b90e1d0df8134a56981dd860a74ddaf4" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 38155 } } peers { permanent_uuid: "ae0b8ab8e3e34b6ca52a2c2904626af0" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 38647 } } peers { permanent_uuid: "f84301c7610748da896c51037f844983" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 39537 } }
I20260809 09:32:50.144068 28701 leader_election.cc:288] T 97e424f0af764ef1a1a0b15854ec6553 P b90e1d0df8134a56981dd860a74ddaf4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ae0b8ab8e3e34b6ca52a2c2904626af0 (127.20.125.129:38647), f84301c7610748da896c51037f844983 (127.20.125.131:39537)
I20260809 09:32:50.146034 28700 raft_consensus.cc:482] T 97e424f0af764ef1a1a0b15854ec6553 P ae0b8ab8e3e34b6ca52a2c2904626af0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 09:32:50.146255 28472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:50.144353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.129:38647 (local address 127.20.125.130:57455)
0809 09:32:50.144525 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0809 09:32:50.144546 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 09:32:50.144644 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:32:50.144934 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:32:50.144937 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:32:50.144952 (+ 15us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:32:50.145539 (+ 587us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 09:32:50.145541 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:32:50.145551 (+ 10us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 09:32:50.145612 (+ 61us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 09:32:50.145956 (+ 344us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 09:32:50.145959 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 09:32:50.145963 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 09:32:50.146071 (+ 108us) client_negotiation.cc:240] Negotiation successful
0809 09:32:50.146114 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":66}
I20260809 09:32:50.146237 28700 raft_consensus.cc:504] T 97e424f0af764ef1a1a0b15854ec6553 P ae0b8ab8e3e34b6ca52a2c2904626af0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b90e1d0df8134a56981dd860a74ddaf4" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 38155 } } peers { permanent_uuid: "ae0b8ab8e3e34b6ca52a2c2904626af0" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 38647 } } peers { permanent_uuid: "f84301c7610748da896c51037f844983" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 39537 } }
I20260809 09:32:50.146255 28690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:50.144469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.130:57455 (local address 127.20.125.129:38647)
0809 09:32:50.144618 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0809 09:32:50.144623 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 09:32:50.144640 (+ 17us) server_negotiation.cc:407] Connection header received
0809 09:32:50.144751 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:32:50.144755 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:32:50.144802 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:32:50.144898 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:32:50.145770 (+ 872us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:32:50.145778 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:32:50.145783 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 09:32:50.145803 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:32:50.145827 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 09:32:50.145839 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:32:50.145960 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0809 09:32:50.146072 (+ 112us) server_negotiation.cc:299] Negotiation successful
0809 09:32:50.146115 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 09:32:50.146642 28689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:50.144578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.130:49579 (local address 127.20.125.131:39537)
0809 09:32:50.144737 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0809 09:32:50.144742 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 09:32:50.145065 (+ 323us) server_negotiation.cc:407] Connection header received
0809 09:32:50.145136 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:32:50.145141 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:32:50.145192 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:32:50.145267 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:32:50.146066 (+ 799us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:32:50.146072 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:32:50.146105 (+ 33us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 09:32:50.146123 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:32:50.146149 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 09:32:50.146162 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:32:50.146236 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0809 09:32:50.146434 (+ 198us) server_negotiation.cc:299] Negotiation successful
0809 09:32:50.146509 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260809 09:32:50.147055 28702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:50.144353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.131:39537 (local address 127.20.125.130:49579)
0809 09:32:50.144955 (+ 602us) negotiation.cc:107] Waiting for socket to connect
0809 09:32:50.144971 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 09:32:50.145058 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:32:50.145325 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:32:50.145330 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:32:50.145350 (+ 20us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:32:50.145860 (+ 510us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 09:32:50.145862 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:32:50.145873 (+ 11us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 09:32:50.145901 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 09:32:50.146322 (+ 421us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 09:32:50.146325 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 09:32:50.146327 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 09:32:50.146859 (+ 532us) client_negotiation.cc:240] Negotiation successful
0809 09:32:50.146902 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":524,"mutex_wait_us":133,"thread_start_us":180,"threads_started":1}
I20260809 09:32:50.147172 28207 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97e424f0af764ef1a1a0b15854ec6553" candidate_uuid: "b90e1d0df8134a56981dd860a74ddaf4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ae0b8ab8e3e34b6ca52a2c2904626af0" is_pre_election: true
I20260809 09:32:50.147418 28207 raft_consensus.cc:2454] T 97e424f0af764ef1a1a0b15854ec6553 P ae0b8ab8e3e34b6ca52a2c2904626af0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b90e1d0df8134a56981dd860a74ddaf4 in term 0.
I20260809 09:32:50.147666 28700 leader_election.cc:288] T 97e424f0af764ef1a1a0b15854ec6553 P ae0b8ab8e3e34b6ca52a2c2904626af0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b90e1d0df8134a56981dd860a74ddaf4 (127.20.125.130:38155), f84301c7610748da896c51037f844983 (127.20.125.131:39537)
I20260809 09:32:50.147797 28593 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97e424f0af764ef1a1a0b15854ec6553" candidate_uuid: "b90e1d0df8134a56981dd860a74ddaf4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f84301c7610748da896c51037f844983" is_pre_election: true
I20260809 09:32:50.148097 28593 raft_consensus.cc:2454] T 97e424f0af764ef1a1a0b15854ec6553 P f84301c7610748da896c51037f844983 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b90e1d0df8134a56981dd860a74ddaf4 in term 0.
I20260809 09:32:50.148169 28306 leader_election.cc:302] T 97e424f0af764ef1a1a0b15854ec6553 P b90e1d0df8134a56981dd860a74ddaf4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ae0b8ab8e3e34b6ca52a2c2904626af0, b90e1d0df8134a56981dd860a74ddaf4; no voters:
I20260809 09:32:50.148484 28701 raft_consensus.cc:2781] T 97e424f0af764ef1a1a0b15854ec6553 P b90e1d0df8134a56981dd860a74ddaf4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 09:32:50.148582 28701 raft_consensus.cc:482] T 97e424f0af764ef1a1a0b15854ec6553 P b90e1d0df8134a56981dd860a74ddaf4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 09:32:50.148643 28701 raft_consensus.cc:3037] T 97e424f0af764ef1a1a0b15854ec6553 P b90e1d0df8134a56981dd860a74ddaf4 [term 0 FOLLOWER]: Advancing to term 1
I20260809 09:32:50.149600 28284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:50.147792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.130:38155 (local address 127.20.125.129:38783)
0809 09:32:50.148047 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0809 09:32:50.148065 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 09:32:50.148155 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:32:50.148508 (+ 353us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:32:50.148515 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:32:50.148536 (+ 21us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:32:50.149012 (+ 476us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 09:32:50.149017 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:32:50.149031 (+ 14us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 09:32:50.149060 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 09:32:50.149345 (+ 285us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 09:32:50.149348 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 09:32:50.149352 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 09:32:50.149430 (+ 78us) client_negotiation.cc:240] Negotiation successful
0809 09:32:50.149483 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":180}
I20260809 09:32:50.149605 28682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:50.147876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.129:38783 (local address 127.20.125.130:38155)
0809 09:32:50.147986 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0809 09:32:50.147990 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 09:32:50.148248 (+ 258us) server_negotiation.cc:407] Connection header received
0809 09:32:50.148292 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:32:50.148299 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:32:50.148336 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:32:50.148411 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:32:50.149198 (+ 787us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:32:50.149204 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:32:50.149207 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 09:32:50.149223 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:32:50.149245 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 09:32:50.149258 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:32:50.149347 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0809 09:32:50.149441 (+ 94us) server_negotiation.cc:299] Negotiation successful
0809 09:32:50.149480 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260809 09:32:50.150322 28405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97e424f0af764ef1a1a0b15854ec6553" candidate_uuid: "ae0b8ab8e3e34b6ca52a2c2904626af0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b90e1d0df8134a56981dd860a74ddaf4" is_pre_election: true
I20260809 09:32:50.150645 28689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:50.148378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.129:50857 (local address 127.20.125.131:39537)
0809 09:32:50.148501 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0809 09:32:50.148505 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 09:32:50.148818 (+ 313us) server_negotiation.cc:407] Connection header received
0809 09:32:50.148900 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:32:50.148906 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:32:50.148945 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:32:50.149041 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:32:50.150187 (+ 1146us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:32:50.150191 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:32:50.150194 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 09:32:50.150206 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:32:50.150229 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 09:32:50.150238 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:32:50.150335 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0809 09:32:50.150446 (+ 111us) server_negotiation.cc:299] Negotiation successful
0809 09:32:50.150517 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260809 09:32:50.150643 28703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:50.147951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.131:39537 (local address 127.20.125.129:50857)
0809 09:32:50.148708 (+ 757us) negotiation.cc:107] Waiting for socket to connect
0809 09:32:50.148726 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 09:32:50.148818 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:32:50.149070 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:32:50.149087 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:32:50.149109 (+ 22us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:32:50.150030 (+ 921us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 09:32:50.150032 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:32:50.150043 (+ 11us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 09:32:50.150071 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 09:32:50.150326 (+ 255us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 09:32:50.150329 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 09:32:50.150332 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 09:32:50.150450 (+ 118us) client_negotiation.cc:240] Negotiation successful
0809 09:32:50.150513 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":670,"thread_start_us":132,"threads_started":1}
I20260809 09:32:50.150607 28701 raft_consensus.cc:504] T 97e424f0af764ef1a1a0b15854ec6553 P b90e1d0df8134a56981dd860a74ddaf4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b90e1d0df8134a56981dd860a74ddaf4" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 38155 } } peers { permanent_uuid: "ae0b8ab8e3e34b6ca52a2c2904626af0" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 38647 } } peers { permanent_uuid: "f84301c7610748da896c51037f844983" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 39537 } }
I20260809 09:32:50.150846 28405 raft_consensus.cc:2379] T 97e424f0af764ef1a1a0b15854ec6553 P b90e1d0df8134a56981dd860a74ddaf4 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ae0b8ab8e3e34b6ca52a2c2904626af0 in current term 1: Already voted for candidate b90e1d0df8134a56981dd860a74ddaf4 in this term.
I20260809 09:32:50.150996 28701 leader_election.cc:288] T 97e424f0af764ef1a1a0b15854ec6553 P b90e1d0df8134a56981dd860a74ddaf4 [CANDIDATE]: Term 1 election: Requested vote from peers ae0b8ab8e3e34b6ca52a2c2904626af0 (127.20.125.129:38647), f84301c7610748da896c51037f844983 (127.20.125.131:39537)
I20260809 09:32:50.151465 28207 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97e424f0af764ef1a1a0b15854ec6553" candidate_uuid: "b90e1d0df8134a56981dd860a74ddaf4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ae0b8ab8e3e34b6ca52a2c2904626af0"
I20260809 09:32:50.151590 28593 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97e424f0af764ef1a1a0b15854ec6553" candidate_uuid: "ae0b8ab8e3e34b6ca52a2c2904626af0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f84301c7610748da896c51037f844983" is_pre_election: true
I20260809 09:32:50.151621 28207 raft_consensus.cc:3037] T 97e424f0af764ef1a1a0b15854ec6553 P ae0b8ab8e3e34b6ca52a2c2904626af0 [term 0 FOLLOWER]: Advancing to term 1
I20260809 09:32:50.151732 28593 raft_consensus.cc:2454] T 97e424f0af764ef1a1a0b15854ec6553 P f84301c7610748da896c51037f844983 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ae0b8ab8e3e34b6ca52a2c2904626af0 in term 0.
I20260809 09:32:50.152104 28111 leader_election.cc:302] T 97e424f0af764ef1a1a0b15854ec6553 P ae0b8ab8e3e34b6ca52a2c2904626af0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ae0b8ab8e3e34b6ca52a2c2904626af0, f84301c7610748da896c51037f844983; no voters: b90e1d0df8134a56981dd860a74ddaf4
I20260809 09:32:50.153028 28207 raft_consensus.cc:2454] T 97e424f0af764ef1a1a0b15854ec6553 P ae0b8ab8e3e34b6ca52a2c2904626af0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b90e1d0df8134a56981dd860a74ddaf4 in term 1.
I20260809 09:32:50.153141 28700 raft_consensus.cc:2741] T 97e424f0af764ef1a1a0b15854ec6553 P ae0b8ab8e3e34b6ca52a2c2904626af0 [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260809 09:32:50.153335 28592 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97e424f0af764ef1a1a0b15854ec6553" candidate_uuid: "b90e1d0df8134a56981dd860a74ddaf4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f84301c7610748da896c51037f844983"
I20260809 09:32:50.153486 28306 leader_election.cc:302] T 97e424f0af764ef1a1a0b15854ec6553 P b90e1d0df8134a56981dd860a74ddaf4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ae0b8ab8e3e34b6ca52a2c2904626af0, b90e1d0df8134a56981dd860a74ddaf4; no voters:
I20260809 09:32:50.153496 28592 raft_consensus.cc:3037] T 97e424f0af764ef1a1a0b15854ec6553 P f84301c7610748da896c51037f844983 [term 0 FOLLOWER]: Advancing to term 1
I20260809 09:32:50.153684 28701 raft_consensus.cc:2781] T 97e424f0af764ef1a1a0b15854ec6553 P b90e1d0df8134a56981dd860a74ddaf4 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 09:32:50.154004 28701 raft_consensus.cc:686] T 97e424f0af764ef1a1a0b15854ec6553 P b90e1d0df8134a56981dd860a74ddaf4 [term 1 LEADER]: Becoming Leader. State: Replica: b90e1d0df8134a56981dd860a74ddaf4, State: Running, Role: LEADER
I20260809 09:32:50.154345 28701 consensus_queue.cc:237] T 97e424f0af764ef1a1a0b15854ec6553 P b90e1d0df8134a56981dd860a74ddaf4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b90e1d0df8134a56981dd860a74ddaf4" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 38155 } } peers { permanent_uuid: "ae0b8ab8e3e34b6ca52a2c2904626af0" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 38647 } } peers { permanent_uuid: "f84301c7610748da896c51037f844983" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 39537 } }
I20260809 09:32:50.154887 28592 raft_consensus.cc:2454] T 97e424f0af764ef1a1a0b15854ec6553 P f84301c7610748da896c51037f844983 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b90e1d0df8134a56981dd860a74ddaf4 in term 1.
I20260809 09:32:50.158306 27987 catalog_manager.cc:5495] T 97e424f0af764ef1a1a0b15854ec6553 P b90e1d0df8134a56981dd860a74ddaf4 reported cstate change: term changed from 0 to 1, leader changed from <none> to b90e1d0df8134a56981dd860a74ddaf4 (127.20.125.130). New cstate: current_term: 1 leader_uuid: "b90e1d0df8134a56981dd860a74ddaf4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b90e1d0df8134a56981dd860a74ddaf4" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 38155 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ae0b8ab8e3e34b6ca52a2c2904626af0" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 38647 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f84301c7610748da896c51037f844983" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 39537 } health_report { overall_health: UNKNOWN } } }
W20260809 09:32:50.171903 28282 tablet.cc:2369] T 97e424f0af764ef1a1a0b15854ec6553 P ae0b8ab8e3e34b6ca52a2c2904626af0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 09:32:50.178462 28682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:50.176879 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44424 (local address 127.20.125.130:38155)
0809 09:32:50.177023 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0809 09:32:50.177027 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 09:32:50.177042 (+ 15us) server_negotiation.cc:407] Connection header received
0809 09:32:50.177186 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:32:50.177188 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:32:50.177252 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:32:50.177324 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:32:50.178047 (+ 723us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:32:50.178050 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:32:50.178052 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 09:32:50.178061 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:32:50.178079 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 09:32:50.178088 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:32:50.178162 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0809 09:32:50.178244 (+ 82us) server_negotiation.cc:299] Negotiation successful
0809 09:32:50.178281 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 09:32:50.184499 28207 raft_consensus.cc:1261] T 97e424f0af764ef1a1a0b15854ec6553 P ae0b8ab8e3e34b6ca52a2c2904626af0 [term 1 FOLLOWER]: Refusing update from remote peer b90e1d0df8134a56981dd860a74ddaf4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 09:32:50.184509 28592 raft_consensus.cc:1261] T 97e424f0af764ef1a1a0b15854ec6553 P f84301c7610748da896c51037f844983 [term 1 FOLLOWER]: Refusing update from remote peer b90e1d0df8134a56981dd860a74ddaf4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 09:32:50.185168 28701 consensus_queue.cc:1035] T 97e424f0af764ef1a1a0b15854ec6553 P b90e1d0df8134a56981dd860a74ddaf4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ae0b8ab8e3e34b6ca52a2c2904626af0" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 38647 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 09:32:50.185389 28704 consensus_queue.cc:1035] T 97e424f0af764ef1a1a0b15854ec6553 P b90e1d0df8134a56981dd860a74ddaf4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f84301c7610748da896c51037f844983" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 39537 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 09:32:50.193354 28709 mvcc.cc:205] Tried to move back new op lower bound from 7316553605868449792 to 7316553605757108224. Current Snapshot: MvccSnapshot[applied={T|T < 7316553605868449792}]
I20260809 09:32:50.204855 27989 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:48662:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 09:32:50.205025 27989 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:48662:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 09:32:50.207628 27989 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d6e9b8159ab94bc3b57260a9c9d80ec6: Sending DeleteTablet for 3 replicas of tablet 97e424f0af764ef1a1a0b15854ec6553
I20260809 09:32:50.208521 28566 tablet_service.cc:1514] Processing DeleteTablet for tablet 97e424f0af764ef1a1a0b15854ec6553 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 09:32:50 UTC) from {username='slave'} at 127.0.0.1:43388
I20260809 09:32:50.208612 28156 tablet_service.cc:1514] Processing DeleteTablet for tablet 97e424f0af764ef1a1a0b15854ec6553 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 09:32:50 UTC) from {username='slave'} at 127.0.0.1:52334
I20260809 09:32:50.208719 28376 tablet_service.cc:1514] Processing DeleteTablet for tablet 97e424f0af764ef1a1a0b15854ec6553 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 09:32:50 UTC) from {username='slave'} at 127.0.0.1:44408
I20260809 09:32:50.209242 28726 tablet_replica.cc:330] T 97e424f0af764ef1a1a0b15854ec6553 P b90e1d0df8134a56981dd860a74ddaf4: stopping tablet replica
I20260809 09:32:50.209335 28724 tablet_replica.cc:330] T 97e424f0af764ef1a1a0b15854ec6553 P f84301c7610748da896c51037f844983: stopping tablet replica
I20260809 09:32:50.209591 28726 raft_consensus.cc:2229] T 97e424f0af764ef1a1a0b15854ec6553 P b90e1d0df8134a56981dd860a74ddaf4 [term 1 LEADER]: Raft consensus shutting down.
I20260809 09:32:50.209795 20982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu with pid 28084
I20260809 09:32:50.209776 28724 raft_consensus.cc:2229] T 97e424f0af764ef1a1a0b15854ec6553 P f84301c7610748da896c51037f844983 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 09:32:50.210096 28726 raft_consensus.cc:2258] T 97e424f0af764ef1a1a0b15854ec6553 P b90e1d0df8134a56981dd860a74ddaf4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 09:32:50.211287 28724 raft_consensus.cc:2258] T 97e424f0af764ef1a1a0b15854ec6553 P f84301c7610748da896c51037f844983 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 09:32:50.216512 28726 ts_tablet_manager.cc:1905] T 97e424f0af764ef1a1a0b15854ec6553 P b90e1d0df8134a56981dd860a74ddaf4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 09:32:50.217429 28724 ts_tablet_manager.cc:1905] T 97e424f0af764ef1a1a0b15854ec6553 P f84301c7610748da896c51037f844983: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 09:32:50.221362 28724 ts_tablet_manager.cc:1918] T 97e424f0af764ef1a1a0b15854ec6553 P f84301c7610748da896c51037f844983: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 09:32:50.221552 28724 log.cc:1196] T 97e424f0af764ef1a1a0b15854ec6553 P f84301c7610748da896c51037f844983: Deleting WAL directory at /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786267941944493-20982-0/minicluster-data/ts-2/wal/wals/97e424f0af764ef1a1a0b15854ec6553
I20260809 09:32:50.221920 28724 ts_tablet_manager.cc:1939] T 97e424f0af764ef1a1a0b15854ec6553 P f84301c7610748da896c51037f844983: Deleting consensus metadata
I20260809 09:32:50.223224 28726 ts_tablet_manager.cc:1918] T 97e424f0af764ef1a1a0b15854ec6553 P b90e1d0df8134a56981dd860a74ddaf4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 09:32:50.223346 28726 log.cc:1196] T 97e424f0af764ef1a1a0b15854ec6553 P b90e1d0df8134a56981dd860a74ddaf4: Deleting WAL directory at /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786267941944493-20982-0/minicluster-data/ts-1/wal/wals/97e424f0af764ef1a1a0b15854ec6553
I20260809 09:32:50.223699 28726 ts_tablet_manager.cc:1939] T 97e424f0af764ef1a1a0b15854ec6553 P b90e1d0df8134a56981dd860a74ddaf4: Deleting consensus metadata
I20260809 09:32:50.223619 27971 catalog_manager.cc:4841] TS f84301c7610748da896c51037f844983 (127.20.125.131:39537): tablet 97e424f0af764ef1a1a0b15854ec6553 (table test-table [id=5f7714d3366c40369068e5aa636a2c5b]) successfully deleted
W20260809 09:32:50.224196 27969 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.20.125.129:38647 (error 108)
I20260809 09:32:50.225160 20982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu with pid 28285
I20260809 09:32:50.225214 27969 catalog_manager.cc:4841] TS b90e1d0df8134a56981dd860a74ddaf4 (127.20.125.130:38155): tablet 97e424f0af764ef1a1a0b15854ec6553 (table test-table [id=5f7714d3366c40369068e5aa636a2c5b]) successfully deleted
I20260809 09:32:50.225461 28688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:50.225069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.129:38647 (local address 127.0.0.1:52344)
0809 09:32:50.225245 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0809 09:32:50.225346 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.129:38647: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260809 09:32:50.232678 27969 catalog_manager.cc:4584] TS ae0b8ab8e3e34b6ca52a2c2904626af0 (127.20.125.129:38647): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 97e424f0af764ef1a1a0b15854ec6553: Network error: Client connection negotiation failed: client connection to 127.20.125.129:38647: connect: Connection refused (error 111)
I20260809 09:32:50.238777 20982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu with pid 28473
I20260809 09:32:50.253057 20982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu with pid 27946
2026-08-09T09:32:50Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (1047 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260809 09:32:50.282385 20982 mini_oidc.cc:152] Starting JWKS server
I20260809 09:32:50.284085 20982 webserver.cc:460] Webserver started at https://127.0.0.1:42579/ using document root <none> and password file <none>
I20260809 09:32:50.284278 20982 mini_oidc.cc:189] Starting OIDC Discovery server
I20260809 09:32:50.284564 20982 webserver.cc:460] Webserver started at http://127.0.0.1:40593/ using document root <none> and password file <none>
2026-08-09T09:32:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T09:32:50Z Disabled control of system clock
I20260809 09:32:50.295421 20982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
/tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786267941944493-20982-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786267941944493-20982-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786267941944493-20982-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786267941944493-20982-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.125.190:37549
--webserver_interface=127.20.125.190
--webserver_port=0
--builtin_ntp_servers=127.20.125.148:45805
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.125.190:37549
--enable_jwt_token_auth=true
--jwks_url=https://localhost:42579/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786267941944493-20982-0/testchainca.cert with env {}
W20260809 09:32:50.407684 28736 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 09:32:50.407953 28736 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 09:32:50.407992 28736 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 09:32:50.411993 28736 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 09:32:50.412053 28736 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 09:32:50.412068 28736 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 09:32:50.412086 28736 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 09:32:50.412101 28736 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260809 09:32:50.412115 28736 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260809 09:32:50.412129 28736 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:42579/jwks.json
I20260809 09:32:50.417380 28736 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.125.148:45805
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786267941944493-20982-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786267941944493-20982-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.125.190:37549
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.125.190:37549
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:42579/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786267941944493-20982-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.125.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786267941944493-20982-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28736
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786267941944493-20982-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 08:38:55 UTC on ccycloud.quasar-njjleu.root.comops.site
build id 4575935
I20260809 09:32:50.418344 28736 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 09:32:50.419350 28736 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 09:32:50.427736 28742 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:32:50.427690 28745 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:32:50.427882 28736 server_base.cc:1034] running on GCE node
W20260809 09:32:50.427690 28743 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:32:50.428654 28736 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 09:32:50.429683 28736 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 09:32:50.430843 28736 hybrid_clock.cc:648] HybridClock initialized: now 1786267970430818 us; error 40 us; skew 500 ppm
I20260809 09:32:50.432641 28736 webserver.cc:460] Webserver started at http://127.20.125.190:43307/ using document root <none> and password file <none>
I20260809 09:32:50.433190 28736 fs_manager.cc:359] Metadata directory not provided
I20260809 09:32:50.433251 28736 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 09:32:50.433414 28736 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 09:32:50.435515 28736 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786267941944493-20982-0/minicluster-data/master-0/data/instance:
uuid: "8f026c68f8ed4f6ca662d0c84dcf224f"
format_stamp: "Formatted at 2026-08-09 09:32:50 on dist-test-slave-7kzw"
server_key: "576e4509fb1b73e8fc9c757b5e2c9c52"
server_key_iv: "9c8c0da23e795f25e3d96fcea1cc5e09"
server_key_version: "encryptionkey@0"
I20260809 09:32:50.436031 28736 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786267941944493-20982-0/minicluster-data/master-0/wal/instance:
uuid: "8f026c68f8ed4f6ca662d0c84dcf224f"
format_stamp: "Formatted at 2026-08-09 09:32:50 on dist-test-slave-7kzw"
server_key: "576e4509fb1b73e8fc9c757b5e2c9c52"
server_key_iv: "9c8c0da23e795f25e3d96fcea1cc5e09"
server_key_version: "encryptionkey@0"
I20260809 09:32:50.439718 28736 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260809 09:32:50.442169 28753 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 09:32:50.443468 28736 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 09:32:50.443574 28736 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786267941944493-20982-0/minicluster-data/master-0/data,/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786267941944493-20982-0/minicluster-data/master-0/wal
uuid: "8f026c68f8ed4f6ca662d0c84dcf224f"
format_stamp: "Formatted at 2026-08-09 09:32:50 on dist-test-slave-7kzw"
server_key: "576e4509fb1b73e8fc9c757b5e2c9c52"
server_key_iv: "9c8c0da23e795f25e3d96fcea1cc5e09"
server_key_version: "encryptionkey@0"
I20260809 09:32:50.443665 28736 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786267941944493-20982-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786267941944493-20982-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786267941944493-20982-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 09:32:50.463506 28736 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 09:32:50.464253 28736 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 09:32:50.464457 28736 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 09:32:50.471738 28736 rpc_server.cc:307] RPC server started. Bound to: 127.20.125.190:37549
I20260809 09:32:50.471763 28831 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.125.190:37549 every 8 connection(s)
I20260809 09:32:50.472728 28736 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786267941944493-20982-0/minicluster-data/master-0/data/info.pb
I20260809 09:32:50.475906 28834 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 09:32:50.481554 28834 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8f026c68f8ed4f6ca662d0c84dcf224f: Bootstrap starting.
I20260809 09:32:50.481691 20982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu as pid 28736
I20260809 09:32:50.481830 20982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786267941944493-20982-0/minicluster-data/master-0/wal/instance
I20260809 09:32:50.482124 20982 external_mini_cluster.cc:1428] Setting key 7d446f23d13159c2d6b65f517406b678
I20260809 09:32:50.484402 28834 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8f026c68f8ed4f6ca662d0c84dcf224f: Neither blocks nor log segments found. Creating new log.
I20260809 09:32:50.485126 28834 log.cc:824] T 00000000000000000000000000000000 P 8f026c68f8ed4f6ca662d0c84dcf224f: Log is configured to *not* fsync() on all Append() calls
I20260809 09:32:50.487218 28834 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8f026c68f8ed4f6ca662d0c84dcf224f: No bootstrap required, opened a new log
I20260809 09:32:50.490809 28834 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8f026c68f8ed4f6ca662d0c84dcf224f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f026c68f8ed4f6ca662d0c84dcf224f" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 37549 } }
I20260809 09:32:50.491029 28834 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8f026c68f8ed4f6ca662d0c84dcf224f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 09:32:50.491086 28834 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8f026c68f8ed4f6ca662d0c84dcf224f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f026c68f8ed4f6ca662d0c84dcf224f, State: Initialized, Role: FOLLOWER
I20260809 09:32:50.491578 28839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:50.483490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38468 (local address 127.20.125.190:37549)
0809 09:32:50.484223 (+ 733us) server_negotiation.cc:206] Beginning negotiation
0809 09:32:50.484240 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0809 09:32:50.484277 (+ 37us) server_negotiation.cc:407] Connection header received
0809 09:32:50.484998 (+ 721us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:32:50.485018 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:32:50.485482 (+ 464us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:32:50.485856 (+ 374us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:32:50.486225 (+ 369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:50.487210 (+ 985us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:50.488154 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:50.488638 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:50.488891 (+ 253us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:50.489754 (+ 863us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:32:50.489759 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:32:50.489772 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 09:32:50.490148 (+ 376us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:32:50.490209 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 09:32:50.490229 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:32:50.490331 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0809 09:32:50.490519 (+ 188us) server_negotiation.cc:299] Negotiation successful
0809 09:32:50.490772 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":314,"thread_start_us":154,"threads_started":1}
I20260809 09:32:50.491539 28834 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8f026c68f8ed4f6ca662d0c84dcf224f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f026c68f8ed4f6ca662d0c84dcf224f" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 37549 } }
I20260809 09:32:50.491781 28834 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8f026c68f8ed4f6ca662d0c84dcf224f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 09:32:50.491860 28834 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8f026c68f8ed4f6ca662d0c84dcf224f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 09:32:50.491935 28834 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8f026c68f8ed4f6ca662d0c84dcf224f [term 0 FOLLOWER]: Advancing to term 1
I20260809 09:32:50.492857 28834 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8f026c68f8ed4f6ca662d0c84dcf224f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f026c68f8ed4f6ca662d0c84dcf224f" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 37549 } }
I20260809 09:32:50.493206 28834 leader_election.cc:302] T 00000000000000000000000000000000 P 8f026c68f8ed4f6ca662d0c84dcf224f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8f026c68f8ed4f6ca662d0c84dcf224f; no voters:
I20260809 09:32:50.493572 28834 leader_election.cc:288] T 00000000000000000000000000000000 P 8f026c68f8ed4f6ca662d0c84dcf224f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 09:32:50.493731 28843 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8f026c68f8ed4f6ca662d0c84dcf224f [term 1 FOLLOWER]: Leader election won for term 1
I20260809 09:32:50.494107 28843 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8f026c68f8ed4f6ca662d0c84dcf224f [term 1 LEADER]: Becoming Leader. State: Replica: 8f026c68f8ed4f6ca662d0c84dcf224f, State: Running, Role: LEADER
I20260809 09:32:50.494623 20982 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 09:32:50.494616 28843 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8f026c68f8ed4f6ca662d0c84dcf224f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f026c68f8ed4f6ca662d0c84dcf224f" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 37549 } }
I20260809 09:32:50.494902 28834 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8f026c68f8ed4f6ca662d0c84dcf224f [sys.catalog]: configured and running, proceeding with master startup.
I20260809 09:32:50.496306 28844 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8f026c68f8ed4f6ca662d0c84dcf224f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8f026c68f8ed4f6ca662d0c84dcf224f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f026c68f8ed4f6ca662d0c84dcf224f" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 37549 } } }
I20260809 09:32:50.496439 28844 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8f026c68f8ed4f6ca662d0c84dcf224f [sys.catalog]: This master's current role is: LEADER
I20260809 09:32:50.496773 28846 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8f026c68f8ed4f6ca662d0c84dcf224f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8f026c68f8ed4f6ca662d0c84dcf224f. Latest consensus state: current_term: 1 leader_uuid: "8f026c68f8ed4f6ca662d0c84dcf224f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f026c68f8ed4f6ca662d0c84dcf224f" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 37549 } } }
I20260809 09:32:50.496869 28846 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8f026c68f8ed4f6ca662d0c84dcf224f [sys.catalog]: This master's current role is: LEADER
W20260809 09:32:50.500047 28864 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8f026c68f8ed4f6ca662d0c84dcf224f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 09:32:50.500145 28864 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 09:32:50.500273 28859 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 09:32:50.501222 28859 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 09:32:50.507890 28859 catalog_manager.cc:1347] Generated new cluster ID: d81a5639343342189fe54f07fbe5f9bd
I20260809 09:32:50.507997 28859 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 09:32:50.531761 28859 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 09:32:50.532734 28859 catalog_manager.cc:1504] Loading token signing keys...
I20260809 09:32:50.538450 28859 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8f026c68f8ed4f6ca662d0c84dcf224f: Generated new TSK 0
I20260809 09:32:50.539165 28859 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260809 09:32:50.571067 28878 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260809 09:32:50.571290 28878 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260809 09:32:50.571875 28839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:50.555029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38480 (local address 127.20.125.190:37549)
0809 09:32:50.555184 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0809 09:32:50.555189 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 09:32:50.555389 (+ 200us) server_negotiation.cc:407] Connection header received
0809 09:32:50.555634 (+ 245us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:32:50.555637 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:32:50.555700 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:32:50.555797 (+ 97us) server_negotiation.cc:226] Negotiated authn=JWT
0809 09:32:50.556080 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:50.556619 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:50.557516 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:50.557750 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:50.557857 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:50.558228 (+ 371us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0809 09:32:50.571417 (+ 13189us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42579/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0809 09:32:50.571710 (+ 293us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:38480: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42579/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":69}
W20260809 09:32:50.571971 28839 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:38480: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42579/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260809 09:32:50.573163 28876 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 09:32:50.554924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:37549 (local address 127.0.0.1:38480)
0809 09:32:50.555302 (+ 378us) negotiation.cc:107] Waiting for socket to connect
0809 09:32:50.555314 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 09:32:50.555463 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:32:50.555828 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:32:50.555832 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:32:50.555840 (+ 8us) client_negotiation.cc:189] Negotiated authn=JWT
0809 09:32:50.555962 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:32:50.555968 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:32:50.556745 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:32:50.556749 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:32:50.557389 (+ 640us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:32:50.557397 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:32:50.557876 (+ 479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:32:50.557880 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:32:50.557987 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:50.558032 (+ 45us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0809 09:32:50.571657 (+ 13625us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42579/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0809 09:32:50.571889 (+ 232us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.20.125.190:37549: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42579/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":106,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.20.125.190:37549: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42579/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260809 09:32:50.573894 20982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu with pid 28736
2026-08-09T09:32:50Z chronyd exiting
I20260809 09:32:50.618938 20982 test_util.cc:175] -----------------------------------------------
I20260809 09:32:50.619036 20982 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786267941944493-20982-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (338 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260809 09:32:50.620152 20982 mini_oidc.cc:152] Starting JWKS server
I20260809 09:32:50.621031 20982 webserver.cc:460] Webserver started at https://127.0.0.1:35233/ using document root <none> and password file <none>
I20260809 09:32:50.621140 20982 mini_oidc.cc:189] Starting OIDC Discovery server
I20260809 09:32:50.621455 20982 webserver.cc:460] Webserver started at http://127.0.0.1:37509/ using document root <none> and password file <none>
2026-08-09T09:32:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T09:32:50Z Disabled control of system clock
I20260809 09:32:50.632314 20982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
/tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786267941944493-20982-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786267941944493-20982-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786267941944493-20982-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786267941944493-20982-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.125.190:40229
--webserver_interface=127.20.125.190
--webserver_port=0
--builtin_ntp_servers=127.20.125.148:40251
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.125.190:40229
--enable_jwt_token_auth=true
--jwks_url=https://localhost:35233/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786267941944493-20982-0/testchainca.cert with env {}
W20260809 09:32:50.746956 28885 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 09:32:50.747226 28885 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 09:32:50.747267 28885 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 09:32:50.751362 28885 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 09:32:50.751425 28885 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 09:32:50.751441 28885 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 09:32:50.751457 28885 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 09:32:50.751472 28885 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260809 09:32:50.751487 28885 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260809 09:32:50.751541 28885 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:35233/jwks.json
I20260809 09:32:50.756913 28885 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.125.148:40251
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786267941944493-20982-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786267941944493-20982-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.125.190:40229
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.125.190:40229
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:35233/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786267941944493-20982-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.125.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786267941944493-20982-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28885
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786267941944493-20982-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 08:38:55 UTC on ccycloud.quasar-njjleu.root.comops.site
build id 4575935
I20260809 09:32:50.758330 28885 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 09:32:50.759387 28885 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 09:32:50.768170 28890 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:32:50.768164 28894 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:32:50.768193 28892 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:32:50.768500 28885 server_base.cc:1034] running on GCE node
I20260809 09:32:50.769399 28885 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 09:32:50.770556 28885 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 09:32:50.771764 28885 hybrid_clock.cc:648] HybridClock initialized: now 1786267970771751 us; error 37 us; skew 500 ppm
I20260809 09:32:50.773712 28885 webserver.cc:460] Webserver started at http://127.20.125.190:44669/ using document root <none> and password file <none>
I20260809 09:32:50.774307 28885 fs_manager.cc:359] Metadata directory not provided
I20260809 09:32:50.774400 28885 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 09:32:50.774659 28885 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 09:32:50.776721 28885 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786267941944493-20982-0/minicluster-data/master-0/data/instance:
uuid: "bcf0c67c84c041c2be3dfa1685e999a8"
format_stamp: "Formatted at 2026-08-09 09:32:50 on dist-test-slave-7kzw"
server_key: "9a77e1e0b21db7e358d7805380152d38"
server_key_iv: "2385bb1c3e6661a5c84a6faa60416afd"
server_key_version: "encryptionkey@0"
I20260809 09:32:50.777293 28885 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786267941944493-20982-0/minicluster-data/master-0/wal/instance:
uuid: "bcf0c67c84c041c2be3dfa1685e999a8"
format_stamp: "Formatted at 2026-08-09 09:32:50 on dist-test-slave-7kzw"
server_key: "9a77e1e0b21db7e358d7805380152d38"
server_key_iv: "2385bb1c3e6661a5c84a6faa60416afd"
server_key_version: "encryptionkey@0"
I20260809 09:32:50.781047 28885 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260809 09:32:50.783510 28901 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 09:32:50.784848 28885 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 09:32:50.785033 28885 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786267941944493-20982-0/minicluster-data/master-0/data,/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786267941944493-20982-0/minicluster-data/master-0/wal
uuid: "bcf0c67c84c041c2be3dfa1685e999a8"
format_stamp: "Formatted at 2026-08-09 09:32:50 on dist-test-slave-7kzw"
server_key: "9a77e1e0b21db7e358d7805380152d38"
server_key_iv: "2385bb1c3e6661a5c84a6faa60416afd"
server_key_version: "encryptionkey@0"
I20260809 09:32:50.785197 28885 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786267941944493-20982-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786267941944493-20982-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786267941944493-20982-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 09:32:50.823482 28885 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 09:32:50.824299 28885 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 09:32:50.824532 28885 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 09:32:50.832367 28885 rpc_server.cc:307] RPC server started. Bound to: 127.20.125.190:40229
I20260809 09:32:50.832404 28979 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.125.190:40229 every 8 connection(s)
I20260809 09:32:50.833387 28885 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786267941944493-20982-0/minicluster-data/master-0/data/info.pb
I20260809 09:32:50.836668 28980 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 09:32:50.838833 20982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu as pid 28885
I20260809 09:32:50.838977 20982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786267941944493-20982-0/minicluster-data/master-0/wal/instance
I20260809 09:32:50.839308 20982 external_mini_cluster.cc:1428] Setting key b05dcbca98379dc972fdaa79aa3f0712
I20260809 09:32:50.842682 28980 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bcf0c67c84c041c2be3dfa1685e999a8: Bootstrap starting.
I20260809 09:32:50.845553 28980 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bcf0c67c84c041c2be3dfa1685e999a8: Neither blocks nor log segments found. Creating new log.
I20260809 09:32:50.846549 28980 log.cc:824] T 00000000000000000000000000000000 P bcf0c67c84c041c2be3dfa1685e999a8: Log is configured to *not* fsync() on all Append() calls
I20260809 09:32:50.849054 28980 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bcf0c67c84c041c2be3dfa1685e999a8: No bootstrap required, opened a new log
I20260809 09:32:50.849807 28985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:50.840870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45432 (local address 127.20.125.190:40229)
0809 09:32:50.841533 (+ 663us) server_negotiation.cc:206] Beginning negotiation
0809 09:32:50.841550 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0809 09:32:50.841589 (+ 39us) server_negotiation.cc:407] Connection header received
0809 09:32:50.842260 (+ 671us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:32:50.842277 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:32:50.842756 (+ 479us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:32:50.843098 (+ 342us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:32:50.843468 (+ 370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:50.844759 (+ 1291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:50.846105 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:50.846746 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:50.846892 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:50.847891 (+ 999us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:32:50.847897 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:32:50.847912 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 09:32:50.848350 (+ 438us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:32:50.848415 (+ 65us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 09:32:50.848437 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:32:50.848515 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0809 09:32:50.848689 (+ 174us) server_negotiation.cc:299] Negotiation successful
0809 09:32:50.848989 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":298,"thread_start_us":144,"threads_started":1}
I20260809 09:32:50.853015 20982 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 09:32:50.853348 28980 raft_consensus.cc:348] T 00000000000000000000000000000000 P bcf0c67c84c041c2be3dfa1685e999a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bcf0c67c84c041c2be3dfa1685e999a8" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 40229 } }
I20260809 09:32:50.853569 28980 raft_consensus.cc:374] T 00000000000000000000000000000000 P bcf0c67c84c041c2be3dfa1685e999a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 09:32:50.853658 28980 raft_consensus.cc:729] T 00000000000000000000000000000000 P bcf0c67c84c041c2be3dfa1685e999a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bcf0c67c84c041c2be3dfa1685e999a8, State: Initialized, Role: FOLLOWER
I20260809 09:32:50.854188 28980 consensus_queue.cc:260] T 00000000000000000000000000000000 P bcf0c67c84c041c2be3dfa1685e999a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bcf0c67c84c041c2be3dfa1685e999a8" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 40229 } }
I20260809 09:32:50.854355 28980 raft_consensus.cc:388] T 00000000000000000000000000000000 P bcf0c67c84c041c2be3dfa1685e999a8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 09:32:50.854415 28980 raft_consensus.cc:482] T 00000000000000000000000000000000 P bcf0c67c84c041c2be3dfa1685e999a8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 09:32:50.854537 28980 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bcf0c67c84c041c2be3dfa1685e999a8 [term 0 FOLLOWER]: Advancing to term 1
I20260809 09:32:50.855628 28980 raft_consensus.cc:504] T 00000000000000000000000000000000 P bcf0c67c84c041c2be3dfa1685e999a8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bcf0c67c84c041c2be3dfa1685e999a8" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 40229 } }
I20260809 09:32:50.855927 28980 leader_election.cc:302] T 00000000000000000000000000000000 P bcf0c67c84c041c2be3dfa1685e999a8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bcf0c67c84c041c2be3dfa1685e999a8; no voters:
I20260809 09:32:50.856189 28980 leader_election.cc:288] T 00000000000000000000000000000000 P bcf0c67c84c041c2be3dfa1685e999a8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 09:32:50.856345 28987 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bcf0c67c84c041c2be3dfa1685e999a8 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 09:32:50.856611 28987 raft_consensus.cc:686] T 00000000000000000000000000000000 P bcf0c67c84c041c2be3dfa1685e999a8 [term 1 LEADER]: Becoming Leader. State: Replica: bcf0c67c84c041c2be3dfa1685e999a8, State: Running, Role: LEADER
I20260809 09:32:50.856950 28987 consensus_queue.cc:237] T 00000000000000000000000000000000 P bcf0c67c84c041c2be3dfa1685e999a8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bcf0c67c84c041c2be3dfa1685e999a8" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 40229 } }
I20260809 09:32:50.857466 28980 sys_catalog.cc:564] T 00000000000000000000000000000000 P bcf0c67c84c041c2be3dfa1685e999a8 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 09:32:50.858582 28988 sys_catalog.cc:455] T 00000000000000000000000000000000 P bcf0c67c84c041c2be3dfa1685e999a8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bcf0c67c84c041c2be3dfa1685e999a8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bcf0c67c84c041c2be3dfa1685e999a8" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 40229 } } }
I20260809 09:32:50.858736 28988 sys_catalog.cc:458] T 00000000000000000000000000000000 P bcf0c67c84c041c2be3dfa1685e999a8 [sys.catalog]: This master's current role is: LEADER
I20260809 09:32:50.859071 28989 sys_catalog.cc:455] T 00000000000000000000000000000000 P bcf0c67c84c041c2be3dfa1685e999a8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bcf0c67c84c041c2be3dfa1685e999a8. Latest consensus state: current_term: 1 leader_uuid: "bcf0c67c84c041c2be3dfa1685e999a8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bcf0c67c84c041c2be3dfa1685e999a8" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 40229 } } }
I20260809 09:32:50.859187 28989 sys_catalog.cc:458] T 00000000000000000000000000000000 P bcf0c67c84c041c2be3dfa1685e999a8 [sys.catalog]: This master's current role is: LEADER
W20260809 09:32:50.862663 29007 catalog_manager.cc:1558] T 00000000000000000000000000000000 P bcf0c67c84c041c2be3dfa1685e999a8: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 09:32:50.862733 29007 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 09:32:50.862809 29000 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 09:32:50.863502 29000 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 09:32:50.868731 29000 catalog_manager.cc:1347] Generated new cluster ID: a997b42be6ef430f9868e9d34fdf12f0
I20260809 09:32:50.868799 29000 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 09:32:50.878635 29000 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 09:32:50.879875 29000 catalog_manager.cc:1504] Loading token signing keys...
I20260809 09:32:50.901361 29000 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bcf0c67c84c041c2be3dfa1685e999a8: Generated new TSK 0
I20260809 09:32:50.902127 29000 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260809 09:32:50.931645 29023 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260809 09:32:50.931758 29023 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260809 09:32:50.932456 28985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:50.916132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45446 (local address 127.20.125.190:40229)
0809 09:32:50.916295 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0809 09:32:50.916299 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 09:32:50.916539 (+ 240us) server_negotiation.cc:407] Connection header received
0809 09:32:50.916691 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:32:50.916695 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:32:50.916754 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:32:50.916870 (+ 116us) server_negotiation.cc:226] Negotiated authn=JWT
0809 09:32:50.917083 (+ 213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:50.917918 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:50.919010 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:50.919375 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:50.919483 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:50.919760 (+ 277us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0809 09:32:50.932056 (+ 12296us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35233/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0809 09:32:50.932312 (+ 256us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:45446: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35233/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":73}
W20260809 09:32:50.932528 28985 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:45446: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35233/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260809 09:32:50.932564 29022 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 09:32:50.916078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:40229 (local address 127.0.0.1:45446)
0809 09:32:50.916450 (+ 372us) negotiation.cc:107] Waiting for socket to connect
0809 09:32:50.916461 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 09:32:50.916559 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:32:50.916878 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:32:50.916880 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:32:50.916887 (+ 7us) client_negotiation.cc:189] Negotiated authn=JWT
0809 09:32:50.916981 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:32:50.916986 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:32:50.918079 (+ 1093us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:32:50.918083 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:32:50.918892 (+ 809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:32:50.918902 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:32:50.919486 (+ 584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:32:50.919489 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:32:50.919557 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:50.919579 (+ 22us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0809 09:32:50.932238 (+ 12659us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35233/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0809 09:32:50.932395 (+ 157us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.20.125.190:40229: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35233/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":301,"thread_start_us":125,"threads_started":1}
I20260809 09:32:50.933343 20982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu with pid 28885
2026-08-09T09:32:50Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (358 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260809 09:32:50.978236 20982 mini_oidc.cc:152] Starting JWKS server
I20260809 09:32:50.979187 20982 webserver.cc:460] Webserver started at https://127.0.0.1:34625/ using document root <none> and password file <none>
I20260809 09:32:50.979300 20982 mini_oidc.cc:189] Starting OIDC Discovery server
I20260809 09:32:50.979575 20982 webserver.cc:460] Webserver started at http://127.0.0.1:38177/ using document root <none> and password file <none>
2026-08-09T09:32:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T09:32:50Z Disabled control of system clock
I20260809 09:32:50.990645 20982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
/tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786267941944493-20982-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786267941944493-20982-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786267941944493-20982-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786267941944493-20982-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.125.190:41225
--webserver_interface=127.20.125.190
--webserver_port=0
--builtin_ntp_servers=127.20.125.148:33911
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.125.190:41225
--enable_jwt_token_auth=true
--jwks_url=https://localhost:34625/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260809 09:32:51.108763 29032 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 09:32:51.109030 29032 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 09:32:51.109072 29032 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 09:32:51.112915 29032 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 09:32:51.112972 29032 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 09:32:51.112991 29032 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 09:32:51.113011 29032 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 09:32:51.113029 29032 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260809 09:32:51.113049 29032 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260809 09:32:51.113071 29032 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:34625/jwks.json
I20260809 09:32:51.117713 29032 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.125.148:33911
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786267941944493-20982-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786267941944493-20982-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.125.190:41225
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.125.190:41225
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:34625/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786267941944493-20982-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.125.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29032
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786267941944493-20982-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 08:38:55 UTC on ccycloud.quasar-njjleu.root.comops.site
build id 4575935
I20260809 09:32:51.118717 29032 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 09:32:51.120020 29032 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 09:32:51.128499 29043 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:32:51.128499 29039 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:32:51.128775 29032 server_base.cc:1034] running on GCE node
W20260809 09:32:51.128512 29041 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:32:51.129484 29032 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 09:32:51.130548 29032 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 09:32:51.131742 29032 hybrid_clock.cc:648] HybridClock initialized: now 1786267971131723 us; error 30 us; skew 500 ppm
I20260809 09:32:51.133628 29032 webserver.cc:460] Webserver started at http://127.20.125.190:41961/ using document root <none> and password file <none>
I20260809 09:32:51.134207 29032 fs_manager.cc:359] Metadata directory not provided
I20260809 09:32:51.134274 29032 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 09:32:51.134568 29032 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 09:32:51.136557 29032 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786267941944493-20982-0/minicluster-data/master-0/data/instance:
uuid: "107b04c56483418695f5744794858acb"
format_stamp: "Formatted at 2026-08-09 09:32:51 on dist-test-slave-7kzw"
server_key: "c9dea4e078d400731e1902671df6aa1f"
server_key_iv: "c2de136d47395464f6c8da10c5940b31"
server_key_version: "encryptionkey@0"
I20260809 09:32:51.137125 29032 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786267941944493-20982-0/minicluster-data/master-0/wal/instance:
uuid: "107b04c56483418695f5744794858acb"
format_stamp: "Formatted at 2026-08-09 09:32:51 on dist-test-slave-7kzw"
server_key: "c9dea4e078d400731e1902671df6aa1f"
server_key_iv: "c2de136d47395464f6c8da10c5940b31"
server_key_version: "encryptionkey@0"
I20260809 09:32:51.140741 29032 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.004s
I20260809 09:32:51.143103 29051 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 09:32:51.144274 29032 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260809 09:32:51.144408 29032 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786267941944493-20982-0/minicluster-data/master-0/data,/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786267941944493-20982-0/minicluster-data/master-0/wal
uuid: "107b04c56483418695f5744794858acb"
format_stamp: "Formatted at 2026-08-09 09:32:51 on dist-test-slave-7kzw"
server_key: "c9dea4e078d400731e1902671df6aa1f"
server_key_iv: "c2de136d47395464f6c8da10c5940b31"
server_key_version: "encryptionkey@0"
I20260809 09:32:51.144517 29032 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786267941944493-20982-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786267941944493-20982-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786267941944493-20982-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 09:32:51.206581 29032 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 09:32:51.207384 29032 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 09:32:51.207597 29032 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 09:32:51.214926 29032 rpc_server.cc:307] RPC server started. Bound to: 127.20.125.190:41225
I20260809 09:32:51.214936 29131 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.125.190:41225 every 8 connection(s)
I20260809 09:32:51.215890 29032 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786267941944493-20982-0/minicluster-data/master-0/data/info.pb
I20260809 09:32:51.217876 20982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu as pid 29032
I20260809 09:32:51.218003 20982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786267941944493-20982-0/minicluster-data/master-0/wal/instance
I20260809 09:32:51.218313 20982 external_mini_cluster.cc:1428] Setting key e3f48eca52fe2a593433284d37dc8035
I20260809 09:32:51.219659 29132 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 09:32:51.226521 29132 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 107b04c56483418695f5744794858acb: Bootstrap starting.
I20260809 09:32:51.228884 29138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:51.219725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55304 (local address 127.20.125.190:41225)
0809 09:32:51.220359 (+ 634us) server_negotiation.cc:206] Beginning negotiation
0809 09:32:51.220375 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 09:32:51.220410 (+ 35us) server_negotiation.cc:407] Connection header received
0809 09:32:51.221048 (+ 638us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:32:51.221064 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:32:51.221433 (+ 369us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:32:51.221824 (+ 391us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:32:51.222290 (+ 466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:51.223976 (+ 1686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:51.225235 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:51.225741 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:51.225860 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:51.226806 (+ 946us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:32:51.226812 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:32:51.226828 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 09:32:51.227328 (+ 500us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:32:51.227397 (+ 69us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 09:32:51.227441 (+ 44us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:32:51.227542 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0809 09:32:51.227682 (+ 140us) server_negotiation.cc:299] Negotiation successful
0809 09:32:51.227971 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":284,"thread_start_us":134,"threads_started":1}
I20260809 09:32:51.229313 29132 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 107b04c56483418695f5744794858acb: Neither blocks nor log segments found. Creating new log.
I20260809 09:32:51.230171 29132 log.cc:824] T 00000000000000000000000000000000 P 107b04c56483418695f5744794858acb: Log is configured to *not* fsync() on all Append() calls
I20260809 09:32:51.232159 20982 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 09:32:51.232576 29132 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 107b04c56483418695f5744794858acb: No bootstrap required, opened a new log
I20260809 09:32:51.236689 29132 raft_consensus.cc:348] T 00000000000000000000000000000000 P 107b04c56483418695f5744794858acb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "107b04c56483418695f5744794858acb" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 41225 } }
I20260809 09:32:51.236989 29132 raft_consensus.cc:374] T 00000000000000000000000000000000 P 107b04c56483418695f5744794858acb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 09:32:51.237066 29132 raft_consensus.cc:729] T 00000000000000000000000000000000 P 107b04c56483418695f5744794858acb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 107b04c56483418695f5744794858acb, State: Initialized, Role: FOLLOWER
I20260809 09:32:51.237579 29132 consensus_queue.cc:260] T 00000000000000000000000000000000 P 107b04c56483418695f5744794858acb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "107b04c56483418695f5744794858acb" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 41225 } }
I20260809 09:32:51.237725 29132 raft_consensus.cc:388] T 00000000000000000000000000000000 P 107b04c56483418695f5744794858acb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 09:32:51.237810 29132 raft_consensus.cc:482] T 00000000000000000000000000000000 P 107b04c56483418695f5744794858acb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 09:32:51.237938 29132 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 107b04c56483418695f5744794858acb [term 0 FOLLOWER]: Advancing to term 1
I20260809 09:32:51.239071 29132 raft_consensus.cc:504] T 00000000000000000000000000000000 P 107b04c56483418695f5744794858acb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "107b04c56483418695f5744794858acb" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 41225 } }
I20260809 09:32:51.239444 29132 leader_election.cc:302] T 00000000000000000000000000000000 P 107b04c56483418695f5744794858acb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 107b04c56483418695f5744794858acb; no voters:
I20260809 09:32:51.239827 29132 leader_election.cc:288] T 00000000000000000000000000000000 P 107b04c56483418695f5744794858acb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 09:32:51.239960 29142 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 107b04c56483418695f5744794858acb [term 1 FOLLOWER]: Leader election won for term 1
I20260809 09:32:51.240213 29142 raft_consensus.cc:686] T 00000000000000000000000000000000 P 107b04c56483418695f5744794858acb [term 1 LEADER]: Becoming Leader. State: Replica: 107b04c56483418695f5744794858acb, State: Running, Role: LEADER
I20260809 09:32:51.240530 29142 consensus_queue.cc:237] T 00000000000000000000000000000000 P 107b04c56483418695f5744794858acb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "107b04c56483418695f5744794858acb" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 41225 } }
I20260809 09:32:51.241168 29132 sys_catalog.cc:564] T 00000000000000000000000000000000 P 107b04c56483418695f5744794858acb [sys.catalog]: configured and running, proceeding with master startup.
I20260809 09:32:51.242694 29144 sys_catalog.cc:455] T 00000000000000000000000000000000 P 107b04c56483418695f5744794858acb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 107b04c56483418695f5744794858acb. Latest consensus state: current_term: 1 leader_uuid: "107b04c56483418695f5744794858acb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "107b04c56483418695f5744794858acb" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 41225 } } }
I20260809 09:32:51.242842 29144 sys_catalog.cc:458] T 00000000000000000000000000000000 P 107b04c56483418695f5744794858acb [sys.catalog]: This master's current role is: LEADER
I20260809 09:32:51.243705 29143 sys_catalog.cc:455] T 00000000000000000000000000000000 P 107b04c56483418695f5744794858acb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "107b04c56483418695f5744794858acb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "107b04c56483418695f5744794858acb" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 41225 } } }
I20260809 09:32:51.244107 29143 sys_catalog.cc:458] T 00000000000000000000000000000000 P 107b04c56483418695f5744794858acb [sys.catalog]: This master's current role is: LEADER
W20260809 09:32:51.246184 29167 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 107b04c56483418695f5744794858acb: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 09:32:51.246285 29167 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 09:32:51.246379 29159 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 09:32:51.247145 29159 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 09:32:51.252810 29159 catalog_manager.cc:1347] Generated new cluster ID: f34d97b031f7484a93993ed49ba20d8c
I20260809 09:32:51.252882 29159 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 09:32:51.271142 29159 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 09:32:51.272142 29159 catalog_manager.cc:1504] Loading token signing keys...
I20260809 09:32:51.279003 29159 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 107b04c56483418695f5744794858acb: Generated new TSK 0
I20260809 09:32:51.279735 29159 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 09:32:51.302736 29138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:51.296356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55306 (local address 127.20.125.190:41225)
0809 09:32:51.296500 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0809 09:32:51.296505 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 09:32:51.296662 (+ 157us) server_negotiation.cc:407] Connection header received
0809 09:32:51.296820 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:32:51.296823 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:32:51.296878 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:32:51.296992 (+ 114us) server_negotiation.cc:226] Negotiated authn=JWT
0809 09:32:51.297259 (+ 267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:51.298075 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:51.299093 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:51.299443 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:51.299553 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:51.299953 (+ 400us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0809 09:32:51.302339 (+ 2386us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34625/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0809 09:32:51.302594 (+ 255us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:55306: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34625/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":58}
W20260809 09:32:51.302809 29138 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:55306: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34625/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260809 09:32:51.304437 29179 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 09:32:51.296238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:41225 (local address 127.0.0.1:55306)
0809 09:32:51.296572 (+ 334us) negotiation.cc:107] Waiting for socket to connect
0809 09:32:51.296586 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 09:32:51.296689 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:32:51.297002 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:32:51.297004 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:32:51.297011 (+ 7us) client_negotiation.cc:189] Negotiated authn=JWT
0809 09:32:51.297113 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:32:51.297118 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:32:51.298202 (+ 1084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:32:51.298205 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:32:51.298971 (+ 766us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:32:51.298981 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:32:51.299644 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:32:51.299654 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:32:51.299727 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:51.299754 (+ 27us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0809 09:32:51.302529 (+ 2775us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34625/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0809 09:32:51.304269 (+ 1740us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.20.125.190:41225: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34625/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":222,"mutex_wait_us":1542,"thread_start_us":110,"threads_started":1}
I20260809 09:32:51.305001 20982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu with pid 29032
E20260809 09:32:51.321511 29180 webserver.cc:533] Webserver: sslize failed:
E20260809 09:32:51.321601 29180 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-08-09T09:32:51Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (375 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-09T09:32:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T09:32:51Z Disabled control of system clock
I20260809 09:32:51.364950 20982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
/tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786267941944493-20982-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786267941944493-20982-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786267941944493-20982-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786267941944493-20982-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.125.190:45105
--webserver_interface=127.20.125.190
--webserver_port=0
--builtin_ntp_servers=127.20.125.148:40751
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.125.190:45105
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786267941944493-20982-0/testchainca.cert with env {}
W20260809 09:32:51.480074 29186 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 09:32:51.480412 29186 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 09:32:51.480487 29186 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 09:32:51.484790 29186 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 09:32:51.484891 29186 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 09:32:51.484922 29186 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 09:32:51.484959 29186 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 09:32:51.484997 29186 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260809 09:32:51.485034 29186 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260809 09:32:51.485074 29186 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260809 09:32:51.490571 29186 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.125.148:40751
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786267941944493-20982-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786267941944493-20982-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.125.190:45105
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.125.190:45105
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786267941944493-20982-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.125.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786267941944493-20982-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29186
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786267941944493-20982-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 08:38:55 UTC on ccycloud.quasar-njjleu.root.comops.site
build id 4575935
I20260809 09:32:51.491731 29186 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 09:32:51.492786 29186 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 09:32:51.501415 29198 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:32:51.501401 29199 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:32:51.501375 29201 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:32:51.501534 29186 server_base.cc:1034] running on GCE node
I20260809 09:32:51.502230 29186 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 09:32:51.503355 29186 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 09:32:51.504546 29186 hybrid_clock.cc:648] HybridClock initialized: now 1786267971504529 us; error 30 us; skew 500 ppm
I20260809 09:32:51.510895 29186 webserver.cc:460] Webserver started at http://127.20.125.190:41773/ using document root <none> and password file <none>
I20260809 09:32:51.511523 29186 fs_manager.cc:359] Metadata directory not provided
I20260809 09:32:51.511590 29186 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 09:32:51.511826 29186 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 09:32:51.513944 29186 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786267941944493-20982-0/minicluster-data/master-0/data/instance:
uuid: "6d46d90a4f094f6bb3c742987122682e"
format_stamp: "Formatted at 2026-08-09 09:32:51 on dist-test-slave-7kzw"
server_key: "c1755a380d5ce88f0260b3906c180fcc"
server_key_iv: "89c8d8bc7cc029786b189294e4ff5b49"
server_key_version: "encryptionkey@0"
I20260809 09:32:51.514578 29186 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786267941944493-20982-0/minicluster-data/master-0/wal/instance:
uuid: "6d46d90a4f094f6bb3c742987122682e"
format_stamp: "Formatted at 2026-08-09 09:32:51 on dist-test-slave-7kzw"
server_key: "c1755a380d5ce88f0260b3906c180fcc"
server_key_iv: "89c8d8bc7cc029786b189294e4ff5b49"
server_key_version: "encryptionkey@0"
I20260809 09:32:51.518345 29186 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260809 09:32:51.520798 29211 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 09:32:51.522097 29186 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 09:32:51.522251 29186 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786267941944493-20982-0/minicluster-data/master-0/data,/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786267941944493-20982-0/minicluster-data/master-0/wal
uuid: "6d46d90a4f094f6bb3c742987122682e"
format_stamp: "Formatted at 2026-08-09 09:32:51 on dist-test-slave-7kzw"
server_key: "c1755a380d5ce88f0260b3906c180fcc"
server_key_iv: "89c8d8bc7cc029786b189294e4ff5b49"
server_key_version: "encryptionkey@0"
I20260809 09:32:51.522364 29186 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786267941944493-20982-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786267941944493-20982-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786267941944493-20982-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 09:32:51.540063 29186 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 09:32:51.540777 29186 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 09:32:51.541013 29186 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 09:32:51.548206 29293 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.125.190:45105 every 8 connection(s)
I20260809 09:32:51.548204 29186 rpc_server.cc:307] RPC server started. Bound to: 127.20.125.190:45105
I20260809 09:32:51.549216 29186 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786267941944493-20982-0/minicluster-data/master-0/data/info.pb
I20260809 09:32:51.551306 20982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu as pid 29186
I20260809 09:32:51.551455 20982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786267941944493-20982-0/minicluster-data/master-0/wal/instance
I20260809 09:32:51.551786 20982 external_mini_cluster.cc:1428] Setting key eb5f70122776c2a5284a99ba463225e6
I20260809 09:32:51.552245 29294 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 09:32:51.559345 29294 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6d46d90a4f094f6bb3c742987122682e: Bootstrap starting.
I20260809 09:32:51.561151 29297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:51.553088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46112 (local address 127.20.125.190:45105)
0809 09:32:51.553699 (+ 611us) server_negotiation.cc:206] Beginning negotiation
0809 09:32:51.553714 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 09:32:51.553768 (+ 54us) server_negotiation.cc:407] Connection header received
0809 09:32:51.554401 (+ 633us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:32:51.554417 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:32:51.554849 (+ 432us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:32:51.555138 (+ 289us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:32:51.555475 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:51.556431 (+ 956us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:51.557649 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:51.558239 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:51.558364 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:51.559353 (+ 989us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:32:51.559358 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:32:51.559370 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 09:32:51.559818 (+ 448us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:32:51.559875 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 09:32:51.559897 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:32:51.559974 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0809 09:32:51.560140 (+ 166us) server_negotiation.cc:299] Negotiation successful
0809 09:32:51.560421 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":269,"thread_start_us":142,"threads_started":1}
I20260809 09:32:51.561676 29294 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6d46d90a4f094f6bb3c742987122682e: Neither blocks nor log segments found. Creating new log.
I20260809 09:32:51.562539 29294 log.cc:824] T 00000000000000000000000000000000 P 6d46d90a4f094f6bb3c742987122682e: Log is configured to *not* fsync() on all Append() calls
I20260809 09:32:51.564301 20982 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 09:32:51.564819 29294 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6d46d90a4f094f6bb3c742987122682e: No bootstrap required, opened a new log
I20260809 09:32:51.568528 29294 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6d46d90a4f094f6bb3c742987122682e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d46d90a4f094f6bb3c742987122682e" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 45105 } }
I20260809 09:32:51.568747 29294 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6d46d90a4f094f6bb3c742987122682e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 09:32:51.568784 29294 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6d46d90a4f094f6bb3c742987122682e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d46d90a4f094f6bb3c742987122682e, State: Initialized, Role: FOLLOWER
I20260809 09:32:51.569130 29294 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6d46d90a4f094f6bb3c742987122682e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d46d90a4f094f6bb3c742987122682e" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 45105 } }
I20260809 09:32:51.569248 29294 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6d46d90a4f094f6bb3c742987122682e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 09:32:51.569288 29294 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6d46d90a4f094f6bb3c742987122682e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 09:32:51.569356 29294 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6d46d90a4f094f6bb3c742987122682e [term 0 FOLLOWER]: Advancing to term 1
I20260809 09:32:51.570197 29294 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6d46d90a4f094f6bb3c742987122682e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d46d90a4f094f6bb3c742987122682e" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 45105 } }
I20260809 09:32:51.570536 29294 leader_election.cc:302] T 00000000000000000000000000000000 P 6d46d90a4f094f6bb3c742987122682e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6d46d90a4f094f6bb3c742987122682e; no voters:
I20260809 09:32:51.570799 29294 leader_election.cc:288] T 00000000000000000000000000000000 P 6d46d90a4f094f6bb3c742987122682e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 09:32:51.570977 29300 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6d46d90a4f094f6bb3c742987122682e [term 1 FOLLOWER]: Leader election won for term 1
I20260809 09:32:51.571238 29300 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6d46d90a4f094f6bb3c742987122682e [term 1 LEADER]: Becoming Leader. State: Replica: 6d46d90a4f094f6bb3c742987122682e, State: Running, Role: LEADER
I20260809 09:32:51.571616 29300 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6d46d90a4f094f6bb3c742987122682e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d46d90a4f094f6bb3c742987122682e" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 45105 } }
I20260809 09:32:51.572048 29294 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6d46d90a4f094f6bb3c742987122682e [sys.catalog]: configured and running, proceeding with master startup.
I20260809 09:32:51.573417 29302 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6d46d90a4f094f6bb3c742987122682e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6d46d90a4f094f6bb3c742987122682e. Latest consensus state: current_term: 1 leader_uuid: "6d46d90a4f094f6bb3c742987122682e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d46d90a4f094f6bb3c742987122682e" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 45105 } } }
I20260809 09:32:51.573545 29302 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6d46d90a4f094f6bb3c742987122682e [sys.catalog]: This master's current role is: LEADER
I20260809 09:32:51.573474 29301 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6d46d90a4f094f6bb3c742987122682e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6d46d90a4f094f6bb3c742987122682e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d46d90a4f094f6bb3c742987122682e" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 45105 } } }
I20260809 09:32:51.573617 29301 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6d46d90a4f094f6bb3c742987122682e [sys.catalog]: This master's current role is: LEADER
I20260809 09:32:51.573958 29316 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 09:32:51.577210 29316 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 09:32:51.583074 29316 catalog_manager.cc:1347] Generated new cluster ID: 92c24e0f23294697ac9517c672b31b63
I20260809 09:32:51.583148 29316 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 09:32:51.594143 29316 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 09:32:51.595088 29316 catalog_manager.cc:1504] Loading token signing keys...
I20260809 09:32:51.600853 29316 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6d46d90a4f094f6bb3c742987122682e: Generated new TSK 0
I20260809 09:32:51.601495 29316 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 09:32:52.066733 29297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:51.615572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46120 (local address 127.20.125.190:45105)
0809 09:32:51.615713 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0809 09:32:51.615718 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 09:32:51.615884 (+ 166us) server_negotiation.cc:407] Connection header received
0809 09:32:51.616023 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:32:51.616026 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:32:51.616082 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:32:51.616198 (+ 116us) server_negotiation.cc:226] Negotiated authn=JWT
0809 09:32:51.616437 (+ 239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:51.617268 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:51.618132 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:51.618517 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:51.618632 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:51.618933 (+ 301us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0809 09:32:52.066252 (+447319us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0809 09:32:52.066601 (+ 349us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:46120: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":58}
W20260809 09:32:52.066811 29297 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:46120: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260809 09:32:52.067226 29335 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 09:32:51.615502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:45105 (local address 127.0.0.1:46120)
0809 09:32:51.615788 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0809 09:32:51.615810 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 09:32:51.615913 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:32:51.616205 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:32:51.616224 (+ 19us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:32:51.616231 (+ 7us) client_negotiation.cc:189] Negotiated authn=JWT
0809 09:32:51.616332 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:32:51.616337 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:32:51.617403 (+ 1066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:32:51.617405 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:32:51.618023 (+ 618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:32:51.618030 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:32:51.618646 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:32:51.618649 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:32:51.618726 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:51.618754 (+ 28us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0809 09:32:52.066553 (+447799us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0809 09:32:52.066738 (+ 185us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.20.125.190:45105: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":108,"threads_started":1}
I20260809 09:32:52.067950 20982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu with pid 29186
2026-08-09T09:32:52Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (742 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (126 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (127 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786267941944493-20982-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 09:32:52 dist-test-slave-7kzw krb5kdc[29349](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 09:32:52 dist-test-slave-7kzw krb5kdc[29349](info): set up 2 sockets
Aug 09 09:32:52 dist-test-slave-7kzw krb5kdc[29349](info): commencing operation
krb5kdc: starting...
W20260809 09:32:53.582383 20982 mini_kdc.cc:121] Time spent starting KDC: real 1.233s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786267941944493-20982-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786267941944493-20982-0/krb5kdc/test-user.keytab.
Aug 09 09:32:53 dist-test-slave-7kzw krb5kdc[29349](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786267973, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T09:32:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T09:32:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.125.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.125.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.125.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.190.keytab.
Entry for principal kudu/127.20.125.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.190.keytab.
I20260809 09:32:53.722203 20982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
/tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786267941944493-20982-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786267941944493-20982-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786267941944493-20982-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786267941944493-20982-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.125.190:33605
--webserver_interface=127.20.125.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.190.keytab
--principal=kudu/127.20.125.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.125.148:39001
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.125.190:33605
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786267941944493-20982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786267941944493-20982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786267941944493-20982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 09:32:53.838945 29368 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 09:32:53.839293 29368 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 09:32:53.839381 29368 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 09:32:53.843470 29368 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 09:32:53.843561 29368 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 09:32:53.843598 29368 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 09:32:53.843649 29368 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 09:32:53.843686 29368 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 09:32:53.849215 29368 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.125.148:39001
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786267941944493-20982-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786267941944493-20982-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.125.190:33605
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.190.keytab
--principal=kudu/127.20.125.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.125.190:33605
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786267941944493-20982-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.125.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29368
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786267941944493-20982-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 08:38:55 UTC on ccycloud.quasar-njjleu.root.comops.site
build id 4575935
I20260809 09:32:53.850360 29368 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 09:32:53.851465 29368 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 09:32:53.860718 29374 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:32:53.860718 29377 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:32:53.860698 29381 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:32:53.860934 29368 server_base.cc:1034] running on GCE node
I20260809 09:32:53.861656 29368 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 09:32:53.862912 29368 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 09:32:53.864089 29368 hybrid_clock.cc:648] HybridClock initialized: now 1786267973864055 us; error 46 us; skew 500 ppm
Aug 09 09:32:53 dist-test-slave-7kzw krb5kdc[29349](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786267973, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 09:32:53.867167 29368 init.cc:377] Logged in from keytab as kudu/127.20.125.190@KRBTEST.COM (short username kudu)
I20260809 09:32:53.868196 29368 webserver.cc:460] Webserver started at http://127.20.125.190:33335/ using document root <none> and password file <none>
I20260809 09:32:53.868803 29368 fs_manager.cc:359] Metadata directory not provided
I20260809 09:32:53.868903 29368 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 09:32:53.869149 29368 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 09:32:53.871387 29368 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786267941944493-20982-0/minicluster-data/master-0/data/instance:
uuid: "a0554e321cc04f71b4063f9ddcb0c7c2"
format_stamp: "Formatted at 2026-08-09 09:32:53 on dist-test-slave-7kzw"
server_key: "a8d0f93177c0cb4e3c6a3f5df2ccc30d"
server_key_iv: "80207ccb7049dc526f1d5fc8267131a8"
server_key_version: "encryptionkey@0"
I20260809 09:32:53.871989 29368 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786267941944493-20982-0/minicluster-data/master-0/wal/instance:
uuid: "a0554e321cc04f71b4063f9ddcb0c7c2"
format_stamp: "Formatted at 2026-08-09 09:32:53 on dist-test-slave-7kzw"
server_key: "a8d0f93177c0cb4e3c6a3f5df2ccc30d"
server_key_iv: "80207ccb7049dc526f1d5fc8267131a8"
server_key_version: "encryptionkey@0"
I20260809 09:32:53.875958 29368 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260809 09:32:53.878669 29390 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 09:32:53.880079 29368 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 09:32:53.880235 29368 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786267941944493-20982-0/minicluster-data/master-0/data,/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786267941944493-20982-0/minicluster-data/master-0/wal
uuid: "a0554e321cc04f71b4063f9ddcb0c7c2"
format_stamp: "Formatted at 2026-08-09 09:32:53 on dist-test-slave-7kzw"
server_key: "a8d0f93177c0cb4e3c6a3f5df2ccc30d"
server_key_iv: "80207ccb7049dc526f1d5fc8267131a8"
server_key_version: "encryptionkey@0"
I20260809 09:32:53.880358 29368 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786267941944493-20982-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786267941944493-20982-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786267941944493-20982-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 09:32:53.897336 29368 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 09:32:53.900918 29368 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 09:32:53.901142 29368 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 09:32:53.911139 29368 rpc_server.cc:307] RPC server started. Bound to: 127.20.125.190:33605
I20260809 09:32:53.911157 29462 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.125.190:33605 every 8 connection(s)
I20260809 09:32:53.912195 29368 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786267941944493-20982-0/minicluster-data/master-0/data/info.pb
I20260809 09:32:53.915019 29463 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 09:32:53.920831 29463 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a0554e321cc04f71b4063f9ddcb0c7c2: Bootstrap starting.
I20260809 09:32:53.921379 20982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu as pid 29368
I20260809 09:32:53.921482 20982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786267941944493-20982-0/minicluster-data/master-0/wal/instance
I20260809 09:32:53.921798 20982 external_mini_cluster.cc:1428] Setting key 82fad31b5deae16416401577d8e6e927
I20260809 09:32:53.923750 29463 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a0554e321cc04f71b4063f9ddcb0c7c2: Neither blocks nor log segments found. Creating new log.
I20260809 09:32:53.924685 29463 log.cc:824] T 00000000000000000000000000000000 P a0554e321cc04f71b4063f9ddcb0c7c2: Log is configured to *not* fsync() on all Append() calls
I20260809 09:32:53.926771 29463 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a0554e321cc04f71b4063f9ddcb0c7c2: No bootstrap required, opened a new log
I20260809 09:32:53.930418 29463 raft_consensus.cc:348] T 00000000000000000000000000000000 P a0554e321cc04f71b4063f9ddcb0c7c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0554e321cc04f71b4063f9ddcb0c7c2" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 33605 } }
I20260809 09:32:53.930734 29463 raft_consensus.cc:374] T 00000000000000000000000000000000 P a0554e321cc04f71b4063f9ddcb0c7c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 09:32:53.930821 29463 raft_consensus.cc:729] T 00000000000000000000000000000000 P a0554e321cc04f71b4063f9ddcb0c7c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0554e321cc04f71b4063f9ddcb0c7c2, State: Initialized, Role: FOLLOWER
I20260809 09:32:53.931317 29463 consensus_queue.cc:260] T 00000000000000000000000000000000 P a0554e321cc04f71b4063f9ddcb0c7c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0554e321cc04f71b4063f9ddcb0c7c2" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 33605 } }
I20260809 09:32:53.931473 29463 raft_consensus.cc:388] T 00000000000000000000000000000000 P a0554e321cc04f71b4063f9ddcb0c7c2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 09:32:53.931546 29463 raft_consensus.cc:482] T 00000000000000000000000000000000 P a0554e321cc04f71b4063f9ddcb0c7c2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 09:32:53.931648 29463 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a0554e321cc04f71b4063f9ddcb0c7c2 [term 0 FOLLOWER]: Advancing to term 1
Aug 09 09:32:53 dist-test-slave-7kzw krb5kdc[29349](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786267973, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.125.190@KRBTEST.COM
I20260809 09:32:53.932838 29463 raft_consensus.cc:504] T 00000000000000000000000000000000 P a0554e321cc04f71b4063f9ddcb0c7c2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0554e321cc04f71b4063f9ddcb0c7c2" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 33605 } }
I20260809 09:32:53.933213 29463 leader_election.cc:302] T 00000000000000000000000000000000 P a0554e321cc04f71b4063f9ddcb0c7c2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a0554e321cc04f71b4063f9ddcb0c7c2; no voters:
I20260809 09:32:53.933568 29463 leader_election.cc:288] T 00000000000000000000000000000000 P a0554e321cc04f71b4063f9ddcb0c7c2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 09:32:53.933717 29471 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a0554e321cc04f71b4063f9ddcb0c7c2 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 09:32:53.933975 29471 raft_consensus.cc:686] T 00000000000000000000000000000000 P a0554e321cc04f71b4063f9ddcb0c7c2 [term 1 LEADER]: Becoming Leader. State: Replica: a0554e321cc04f71b4063f9ddcb0c7c2, State: Running, Role: LEADER
I20260809 09:32:53.934365 29471 consensus_queue.cc:237] T 00000000000000000000000000000000 P a0554e321cc04f71b4063f9ddcb0c7c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0554e321cc04f71b4063f9ddcb0c7c2" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 33605 } }
I20260809 09:32:53.935045 29463 sys_catalog.cc:564] T 00000000000000000000000000000000 P a0554e321cc04f71b4063f9ddcb0c7c2 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 09:32:53.936324 29473 sys_catalog.cc:455] T 00000000000000000000000000000000 P a0554e321cc04f71b4063f9ddcb0c7c2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a0554e321cc04f71b4063f9ddcb0c7c2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0554e321cc04f71b4063f9ddcb0c7c2" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 33605 } } }
I20260809 09:32:53.936452 29473 sys_catalog.cc:458] T 00000000000000000000000000000000 P a0554e321cc04f71b4063f9ddcb0c7c2 [sys.catalog]: This master's current role is: LEADER
I20260809 09:32:53.936774 29472 sys_catalog.cc:455] T 00000000000000000000000000000000 P a0554e321cc04f71b4063f9ddcb0c7c2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a0554e321cc04f71b4063f9ddcb0c7c2. Latest consensus state: current_term: 1 leader_uuid: "a0554e321cc04f71b4063f9ddcb0c7c2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0554e321cc04f71b4063f9ddcb0c7c2" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 33605 } } }
I20260809 09:32:53.936947 29472 sys_catalog.cc:458] T 00000000000000000000000000000000 P a0554e321cc04f71b4063f9ddcb0c7c2 [sys.catalog]: This master's current role is: LEADER
I20260809 09:32:53.938019 29482 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 09:32:53.939003 29468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:53.923308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42268 (local address 127.20.125.190:33605)
0809 09:32:53.923790 (+ 482us) server_negotiation.cc:206] Beginning negotiation
0809 09:32:53.923812 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0809 09:32:53.923859 (+ 47us) server_negotiation.cc:407] Connection header received
0809 09:32:53.924745 (+ 886us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:32:53.924770 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:32:53.925192 (+ 422us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:32:53.925536 (+ 344us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:32:53.926390 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:53.927498 (+ 1108us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:53.928365 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:53.928854 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:53.929032 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:53.932389 (+ 3357us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:32:53.932395 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:32:53.932410 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:32:53.932440 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:32:53.934749 (+ 2309us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:32:53.935304 (+ 555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:32:53.935311 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:32:53.935316 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:32:53.935469 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:32:53.935848 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:32:53.935852 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:32:53.935854 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:32:53.936469 (+ 615us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:32:53.936680 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0809 09:32:53.937616 (+ 936us) server_negotiation.cc:299] Negotiation successful
0809 09:32:53.937947 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":312,"thread_start_us":130,"threads_started":1}
I20260809 09:32:53.941267 29482 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 09:32:53.946914 29482 catalog_manager.cc:1347] Generated new cluster ID: 1f2a081c42dd4bc9a9d5329c1308b218
I20260809 09:32:53.947007 29482 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 09:32:53.965099 29482 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 09:32:53.966699 29482 catalog_manager.cc:1504] Loading token signing keys...
I20260809 09:32:53.973335 29482 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a0554e321cc04f71b4063f9ddcb0c7c2: Generated new TSK 0
I20260809 09:32:53.974179 29482 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.20.125.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.125.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.125.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.129.keytab.
Entry for principal kudu/127.20.125.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.129.keytab.
I20260809 09:32:54.007218 20982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
/tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786267941944493-20982-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786267941944493-20982-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786267941944493-20982-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786267941944493-20982-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.125.129:0
--local_ip_for_outbound_sockets=127.20.125.129
--webserver_interface=127.20.125.129
--webserver_port=0
--tserver_master_addrs=127.20.125.190:33605
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.129.keytab
--principal=kudu/127.20.125.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.125.148:39001
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786267941944493-20982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786267941944493-20982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786267941944493-20982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 09:32:54.132506 29497 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 09:32:54.132827 29497 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 09:32:54.132874 29497 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 09:32:54.137358 29497 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 09:32:54.137473 29497 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 09:32:54.137581 29497 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.125.129
I20260809 09:32:54.143249 29497 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.125.148:39001
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786267941944493-20982-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786267941944493-20982-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.129.keytab
--principal=kudu/127.20.125.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.125.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786267941944493-20982-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.125.129
--webserver_port=0
--tserver_master_addrs=127.20.125.190:33605
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29497
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.125.129
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786267941944493-20982-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 08:38:55 UTC on ccycloud.quasar-njjleu.root.comops.site
build id 4575935
I20260809 09:32:54.144482 29497 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 09:32:54.145681 29497 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 09:32:54.156273 29504 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:32:54.156273 29509 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:32:54.156319 29497 server_base.cc:1034] running on GCE node
W20260809 09:32:54.156292 29503 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:32:54.157141 29497 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 09:32:54.157895 29497 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 09:32:54.159090 29497 hybrid_clock.cc:648] HybridClock initialized: now 1786267974159071 us; error 30 us; skew 500 ppm
Aug 09 09:32:54 dist-test-slave-7kzw krb5kdc[29349](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786267974, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 09:32:54.162379 29497 init.cc:377] Logged in from keytab as kudu/127.20.125.129@KRBTEST.COM (short username kudu)
I20260809 09:32:54.163364 29497 webserver.cc:460] Webserver started at http://127.20.125.129:38239/ using document root <none> and password file <none>
I20260809 09:32:54.163973 29497 fs_manager.cc:359] Metadata directory not provided
I20260809 09:32:54.164073 29497 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 09:32:54.164335 29497 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 09:32:54.166553 29497 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786267941944493-20982-0/minicluster-data/ts-0/data/instance:
uuid: "01961985ddc1497180209ba20580dc50"
format_stamp: "Formatted at 2026-08-09 09:32:54 on dist-test-slave-7kzw"
server_key: "3eae0a8480703d859cfa4959774f962a"
server_key_iv: "eab7d77c75991a7baf49c53945a5d402"
server_key_version: "encryptionkey@0"
I20260809 09:32:54.167150 29497 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786267941944493-20982-0/minicluster-data/ts-0/wal/instance:
uuid: "01961985ddc1497180209ba20580dc50"
format_stamp: "Formatted at 2026-08-09 09:32:54 on dist-test-slave-7kzw"
server_key: "3eae0a8480703d859cfa4959774f962a"
server_key_iv: "eab7d77c75991a7baf49c53945a5d402"
server_key_version: "encryptionkey@0"
I20260809 09:32:54.171217 29497 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260809 09:32:54.174059 29516 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 09:32:54.175463 29497 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 09:32:54.175642 29497 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786267941944493-20982-0/minicluster-data/ts-0/data,/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786267941944493-20982-0/minicluster-data/ts-0/wal
uuid: "01961985ddc1497180209ba20580dc50"
format_stamp: "Formatted at 2026-08-09 09:32:54 on dist-test-slave-7kzw"
server_key: "3eae0a8480703d859cfa4959774f962a"
server_key_iv: "eab7d77c75991a7baf49c53945a5d402"
server_key_version: "encryptionkey@0"
I20260809 09:32:54.175787 29497 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786267941944493-20982-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786267941944493-20982-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786267941944493-20982-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 09:32:54.187444 29497 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 09:32:54.190774 29497 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 09:32:54.190994 29497 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 09:32:54.191617 29497 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 09:32:54.192495 29497 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 09:32:54.192569 29497 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 09:32:54.192644 29497 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 09:32:54.192700 29497 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 09:32:54.203011 29497 rpc_server.cc:307] RPC server started. Bound to: 127.20.125.129:36661
I20260809 09:32:54.203064 29675 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.125.129:36661 every 8 connection(s)
I20260809 09:32:54.203996 29497 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786267941944493-20982-0/minicluster-data/ts-0/data/info.pb
I20260809 09:32:54.206115 20982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu as pid 29497
I20260809 09:32:54.206317 20982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786267941944493-20982-0/minicluster-data/ts-0/wal/instance
I20260809 09:32:54.206729 20982 external_mini_cluster.cc:1428] Setting key 148420aeaa5a17afb6d063735d65bc00
Aug 09 09:32:54 dist-test-slave-7kzw krb5kdc[29349](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786267974, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.129@KRBTEST.COM for kudu/127.20.125.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.20.125.130@KRBTEST.COM; defaulting to no policy
I20260809 09:32:54.220643 29468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:54.206355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.129:48809 (local address 127.20.125.190:33605)
0809 09:32:54.206541 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0809 09:32:54.206546 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 09:32:54.207661 (+ 1115us) server_negotiation.cc:407] Connection header received
0809 09:32:54.209673 (+ 2012us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:32:54.209676 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:32:54.209722 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:32:54.209794 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:32:54.211738 (+ 1944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:54.212295 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:54.213485 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:54.213739 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:54.213860 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:54.216619 (+ 2759us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:32:54.216622 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:32:54.216625 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:32:54.216659 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:32:54.218337 (+ 1678us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:32:54.218917 (+ 580us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:32:54.218920 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:32:54.218922 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:32:54.219011 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:32:54.219390 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:32:54.219394 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:32:54.219395 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:32:54.219625 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:32:54.219742 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0809 09:32:54.220289 (+ 547us) server_negotiation.cc:299] Negotiation successful
0809 09:32:54.220440 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.125.130@KRBTEST.COM" created.
I20260809 09:32:54.221630 29679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:54.206835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:33605 (local address 127.20.125.129:48809)
0809 09:32:54.207460 (+ 625us) negotiation.cc:107] Waiting for socket to connect
0809 09:32:54.207526 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0809 09:32:54.209392 (+ 1866us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:32:54.210006 (+ 614us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:32:54.210018 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:32:54.210862 (+ 844us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:32:54.211566 (+ 704us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:32:54.211583 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:32:54.212447 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:32:54.212450 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:32:54.213343 (+ 893us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:32:54.213353 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:32:54.213910 (+ 557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:32:54.213914 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:32:54.214087 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:54.214860 (+ 773us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 09:32:54.214879 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:32:54.216431 (+ 1552us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 09:32:54.218496 (+ 2065us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:32:54.218503 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:32:54.218520 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:32:54.218799 (+ 279us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:32:54.219115 (+ 316us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:32:54.219118 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:32:54.219120 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:32:54.219280 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:32:54.219742 (+ 462us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 09:32:54.219751 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 09:32:54.220024 (+ 273us) client_negotiation.cc:769] Sending connection context
0809 09:32:54.220227 (+ 203us) client_negotiation.cc:240] Negotiation successful
0809 09:32:54.220577 (+ 350us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":355,"thread_start_us":132,"threads_started":1}
I20260809 09:32:54.223073 29676 heartbeater.cc:347] Connected to a master server at 127.20.125.190:33605
I20260809 09:32:54.223388 29676 heartbeater.cc:464] Registering TS with master...
I20260809 09:32:54.224113 29676 heartbeater.cc:511] Master 127.20.125.190:33605 requested a full tablet report, sending...
I20260809 09:32:54.225891 29414 ts_manager.cc:194] Registered new tserver with Master: 01961985ddc1497180209ba20580dc50 (127.20.125.129:36661)
I20260809 09:32:54.227087 29414 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.125.129@KRBTEST.COM'} at 127.20.125.129:48809
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.125.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.130.keytab.
Entry for principal kudu/127.20.125.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.130.keytab.
I20260809 09:32:54.242205 20982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
/tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786267941944493-20982-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786267941944493-20982-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786267941944493-20982-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786267941944493-20982-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.125.130:0
--local_ip_for_outbound_sockets=127.20.125.130
--webserver_interface=127.20.125.130
--webserver_port=0
--tserver_master_addrs=127.20.125.190:33605
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.130.keytab
--principal=kudu/127.20.125.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.125.148:39001
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786267941944493-20982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786267941944493-20982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786267941944493-20982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 09:32:54.358335 29683 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 09:32:54.358611 29683 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 09:32:54.358661 29683 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 09:32:54.362773 29683 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 09:32:54.362835 29683 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 09:32:54.362893 29683 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.125.130
I20260809 09:32:54.368091 29683 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.125.148:39001
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786267941944493-20982-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786267941944493-20982-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.130.keytab
--principal=kudu/127.20.125.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.125.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786267941944493-20982-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.125.130
--webserver_port=0
--tserver_master_addrs=127.20.125.190:33605
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29683
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.125.130
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786267941944493-20982-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 08:38:55 UTC on ccycloud.quasar-njjleu.root.comops.site
build id 4575935
I20260809 09:32:54.369170 29683 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 09:32:54.370199 29683 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 09:32:54.380157 29689 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:32:54.380136 29692 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:32:54.380256 29696 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:32:54.380360 29683 server_base.cc:1034] running on GCE node
I20260809 09:32:54.380921 29683 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 09:32:54.381604 29683 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 09:32:54.382838 29683 hybrid_clock.cc:648] HybridClock initialized: now 1786267974382795 us; error 56 us; skew 500 ppm
Aug 09 09:32:54 dist-test-slave-7kzw krb5kdc[29349](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786267974, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 09:32:54.385870 29683 init.cc:377] Logged in from keytab as kudu/127.20.125.130@KRBTEST.COM (short username kudu)
I20260809 09:32:54.386891 29683 webserver.cc:460] Webserver started at http://127.20.125.130:46415/ using document root <none> and password file <none>
I20260809 09:32:54.387501 29683 fs_manager.cc:359] Metadata directory not provided
I20260809 09:32:54.387602 29683 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 09:32:54.387848 29683 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 09:32:54.390093 29683 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786267941944493-20982-0/minicluster-data/ts-1/data/instance:
uuid: "1cce2d04d220437397beba899fe0bb85"
format_stamp: "Formatted at 2026-08-09 09:32:54 on dist-test-slave-7kzw"
server_key: "d1691d68397cbe18120736c77fa82585"
server_key_iv: "5105baaa86aab848e9edf6c7e4642a64"
server_key_version: "encryptionkey@0"
I20260809 09:32:54.390744 29683 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786267941944493-20982-0/minicluster-data/ts-1/wal/instance:
uuid: "1cce2d04d220437397beba899fe0bb85"
format_stamp: "Formatted at 2026-08-09 09:32:54 on dist-test-slave-7kzw"
server_key: "d1691d68397cbe18120736c77fa82585"
server_key_iv: "5105baaa86aab848e9edf6c7e4642a64"
server_key_version: "encryptionkey@0"
I20260809 09:32:54.394428 29683 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260809 09:32:54.397082 29706 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 09:32:54.398418 29683 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 09:32:54.398602 29683 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786267941944493-20982-0/minicluster-data/ts-1/data,/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786267941944493-20982-0/minicluster-data/ts-1/wal
uuid: "1cce2d04d220437397beba899fe0bb85"
format_stamp: "Formatted at 2026-08-09 09:32:54 on dist-test-slave-7kzw"
server_key: "d1691d68397cbe18120736c77fa82585"
server_key_iv: "5105baaa86aab848e9edf6c7e4642a64"
server_key_version: "encryptionkey@0"
I20260809 09:32:54.398736 29683 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786267941944493-20982-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786267941944493-20982-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786267941944493-20982-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 09:32:54.436534 29683 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 09:32:54.440220 29683 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 09:32:54.440434 29683 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 09:32:54.441072 29683 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 09:32:54.441947 29683 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 09:32:54.441996 29683 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 09:32:54.442039 29683 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 09:32:54.442093 29683 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 09:32:54.451152 29683 rpc_server.cc:307] RPC server started. Bound to: 127.20.125.130:38379
I20260809 09:32:54.451185 29875 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.125.130:38379 every 8 connection(s)
I20260809 09:32:54.452040 29683 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786267941944493-20982-0/minicluster-data/ts-1/data/info.pb
I20260809 09:32:54.460860 20982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu as pid 29683
I20260809 09:32:54.461016 20982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786267941944493-20982-0/minicluster-data/ts-1/wal/instance
I20260809 09:32:54.461309 20982 external_mini_cluster.cc:1428] Setting key fb43374213569432382d1ced55820faf
Aug 09 09:32:54 dist-test-slave-7kzw krb5kdc[29349](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786267974, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.130@KRBTEST.COM for kudu/127.20.125.190@KRBTEST.COM
I20260809 09:32:54.465752 29468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:54.453876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.130:57465 (local address 127.20.125.190:33605)
0809 09:32:54.454025 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0809 09:32:54.454029 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 09:32:54.454894 (+ 865us) server_negotiation.cc:407] Connection header received
0809 09:32:54.456295 (+ 1401us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:32:54.456299 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:32:54.456356 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:32:54.456458 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:32:54.457589 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:54.458128 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:54.459356 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:54.459578 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:54.459740 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:54.462138 (+ 2398us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:32:54.462141 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:32:54.462146 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:32:54.462177 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:32:54.463709 (+ 1532us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:32:54.464229 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:32:54.464235 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:32:54.464236 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:32:54.464322 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:32:54.464664 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:32:54.464669 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:32:54.464670 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:32:54.464858 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:32:54.464967 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0809 09:32:54.465461 (+ 494us) server_negotiation.cc:299] Negotiation successful
0809 09:32:54.465580 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 09:32:54.466599 29879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:54.454246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:33605 (local address 127.20.125.130:57465)
0809 09:32:54.454751 (+ 505us) negotiation.cc:107] Waiting for socket to connect
0809 09:32:54.454795 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 09:32:54.456049 (+ 1254us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:32:54.456630 (+ 581us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:32:54.456637 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:32:54.456985 (+ 348us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:32:54.457446 (+ 461us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:32:54.457459 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:32:54.458263 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:32:54.458271 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:32:54.459221 (+ 950us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:32:54.459231 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:32:54.459733 (+ 502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:32:54.459737 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:32:54.459917 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:54.460454 (+ 537us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 09:32:54.460471 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:32:54.461974 (+ 1503us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 09:32:54.463832 (+ 1858us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:32:54.463838 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:32:54.463853 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:32:54.464115 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:32:54.464416 (+ 301us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:32:54.464420 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:32:54.464422 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:32:54.464564 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:32:54.464966 (+ 402us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 09:32:54.464973 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 09:32:54.465229 (+ 256us) client_negotiation.cc:769] Sending connection context
0809 09:32:54.465418 (+ 189us) client_negotiation.cc:240] Negotiation successful
0809 09:32:54.465744 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":310,"thread_start_us":163,"threads_started":1}
I20260809 09:32:54.467872 29877 heartbeater.cc:347] Connected to a master server at 127.20.125.190:33605
I20260809 09:32:54.468111 29877 heartbeater.cc:464] Registering TS with master...
I20260809 09:32:54.468578 29877 heartbeater.cc:511] Master 127.20.125.190:33605 requested a full tablet report, sending...
I20260809 09:32:54.469784 29414 ts_manager.cc:194] Registered new tserver with Master: 1cce2d04d220437397beba899fe0bb85 (127.20.125.130:38379)
I20260809 09:32:54.470636 29414 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.125.130@KRBTEST.COM'} at 127.20.125.130:57465
WARNING: no policy specified for kudu/127.20.125.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.125.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.125.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.131.keytab.
Entry for principal kudu/127.20.125.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.131.keytab.
I20260809 09:32:54.493626 20982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
/tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786267941944493-20982-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786267941944493-20982-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786267941944493-20982-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786267941944493-20982-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.125.131:0
--local_ip_for_outbound_sockets=127.20.125.131
--webserver_interface=127.20.125.131
--webserver_port=0
--tserver_master_addrs=127.20.125.190:33605
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.131.keytab
--principal=kudu/127.20.125.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.125.148:39001
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786267941944493-20982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786267941944493-20982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786267941944493-20982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 09:32:54.611789 29883 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 09:32:54.612098 29883 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 09:32:54.612181 29883 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 09:32:54.616261 29883 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 09:32:54.616341 29883 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 09:32:54.616433 29883 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.125.131
I20260809 09:32:54.621685 29883 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.125.148:39001
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786267941944493-20982-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786267941944493-20982-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.131.keytab
--principal=kudu/127.20.125.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.125.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786267941944493-20982-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.125.131
--webserver_port=0
--tserver_master_addrs=127.20.125.190:33605
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29883
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.125.131
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786267941944493-20982-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 08:38:55 UTC on ccycloud.quasar-njjleu.root.comops.site
build id 4575935
I20260809 09:32:54.622838 29883 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 09:32:54.623888 29883 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 09:32:54.633654 29896 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:32:54.633648 29893 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:32:54.633790 29883 server_base.cc:1034] running on GCE node
W20260809 09:32:54.633661 29892 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:32:54.634409 29883 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 09:32:54.635123 29883 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 09:32:54.636292 29883 hybrid_clock.cc:648] HybridClock initialized: now 1786267974636281 us; error 26 us; skew 500 ppm
Aug 09 09:32:54 dist-test-slave-7kzw krb5kdc[29349](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786267974, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 09:32:54.639313 29883 init.cc:377] Logged in from keytab as kudu/127.20.125.131@KRBTEST.COM (short username kudu)
I20260809 09:32:54.640244 29883 webserver.cc:460] Webserver started at http://127.20.125.131:39453/ using document root <none> and password file <none>
I20260809 09:32:54.640852 29883 fs_manager.cc:359] Metadata directory not provided
I20260809 09:32:54.640976 29883 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 09:32:54.641213 29883 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 09:32:54.643402 29883 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786267941944493-20982-0/minicluster-data/ts-2/data/instance:
uuid: "3a9fba53d35a4a848a139aee1bb2a7ce"
format_stamp: "Formatted at 2026-08-09 09:32:54 on dist-test-slave-7kzw"
server_key: "6544b1ca0d1f75e0d560afe33b95aaf6"
server_key_iv: "7a044dfc249290655449b0d4854fe65f"
server_key_version: "encryptionkey@0"
I20260809 09:32:54.643980 29883 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786267941944493-20982-0/minicluster-data/ts-2/wal/instance:
uuid: "3a9fba53d35a4a848a139aee1bb2a7ce"
format_stamp: "Formatted at 2026-08-09 09:32:54 on dist-test-slave-7kzw"
server_key: "6544b1ca0d1f75e0d560afe33b95aaf6"
server_key_iv: "7a044dfc249290655449b0d4854fe65f"
server_key_version: "encryptionkey@0"
I20260809 09:32:54.647603 29883 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.004s
I20260809 09:32:54.650110 29904 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 09:32:54.651531 29883 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 09:32:54.651665 29883 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786267941944493-20982-0/minicluster-data/ts-2/data,/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786267941944493-20982-0/minicluster-data/ts-2/wal
uuid: "3a9fba53d35a4a848a139aee1bb2a7ce"
format_stamp: "Formatted at 2026-08-09 09:32:54 on dist-test-slave-7kzw"
server_key: "6544b1ca0d1f75e0d560afe33b95aaf6"
server_key_iv: "7a044dfc249290655449b0d4854fe65f"
server_key_version: "encryptionkey@0"
I20260809 09:32:54.651834 29883 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786267941944493-20982-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786267941944493-20982-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786267941944493-20982-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 09:32:54.666273 29883 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 09:32:54.669610 29883 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 09:32:54.669828 29883 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 09:32:54.670434 29883 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 09:32:54.671340 29883 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 09:32:54.671412 29883 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 09:32:54.671486 29883 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 09:32:54.671540 29883 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 09:32:54.680788 29883 rpc_server.cc:307] RPC server started. Bound to: 127.20.125.131:43603
I20260809 09:32:54.680853 30073 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.125.131:43603 every 8 connection(s)
I20260809 09:32:54.681758 29883 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786267941944493-20982-0/minicluster-data/ts-2/data/info.pb
Aug 09 09:32:54 dist-test-slave-7kzw krb5kdc[29349](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786267974, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.131@KRBTEST.COM for kudu/127.20.125.190@KRBTEST.COM
I20260809 09:32:54.692150 20982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu as pid 29883
I20260809 09:32:54.692260 20982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786267941944493-20982-0/minicluster-data/ts-2/wal/instance
I20260809 09:32:54.692620 20982 external_mini_cluster.cc:1428] Setting key 4f6e9be027355fcaff4a85c911bf80dc
I20260809 09:32:54.696446 29468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:54.683857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.131:46125 (local address 127.20.125.190:33605)
0809 09:32:54.684041 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0809 09:32:54.684046 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 09:32:54.684869 (+ 823us) server_negotiation.cc:407] Connection header received
0809 09:32:54.686284 (+ 1415us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:32:54.686287 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:32:54.686336 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:32:54.686411 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:32:54.687806 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:54.688403 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:54.689526 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:54.689768 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:54.689860 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:54.692480 (+ 2620us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:32:54.692485 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:32:54.692488 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:32:54.692516 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:32:54.694225 (+ 1709us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:32:54.694880 (+ 655us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:32:54.694884 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:32:54.694886 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:32:54.694975 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:32:54.695333 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:32:54.695336 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:32:54.695338 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:32:54.695528 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:32:54.695639 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0809 09:32:54.696150 (+ 511us) server_negotiation.cc:299] Negotiation successful
0809 09:32:54.696273 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260809 09:32:54.697227 30077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:54.684235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:33605 (local address 127.20.125.131:46125)
0809 09:32:54.684718 (+ 483us) negotiation.cc:107] Waiting for socket to connect
0809 09:32:54.684762 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 09:32:54.686084 (+ 1322us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:32:54.686734 (+ 650us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:32:54.686745 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:32:54.687143 (+ 398us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:32:54.687628 (+ 485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:32:54.687642 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:32:54.688530 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:32:54.688532 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:32:54.689416 (+ 884us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:32:54.689425 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:32:54.689908 (+ 483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:32:54.689910 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:32:54.690047 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:54.690682 (+ 635us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 09:32:54.690699 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:32:54.692266 (+ 1567us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 09:32:54.694361 (+ 2095us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:32:54.694369 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:32:54.694386 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:32:54.694772 (+ 386us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:32:54.695071 (+ 299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:32:54.695075 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:32:54.695077 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:32:54.695225 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:32:54.695646 (+ 421us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 09:32:54.695655 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 09:32:54.695896 (+ 241us) client_negotiation.cc:769] Sending connection context
0809 09:32:54.696079 (+ 183us) client_negotiation.cc:240] Negotiation successful
0809 09:32:54.696406 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":267,"thread_start_us":111,"threads_started":1}
I20260809 09:32:54.698411 30075 heartbeater.cc:347] Connected to a master server at 127.20.125.190:33605
I20260809 09:32:54.698722 30075 heartbeater.cc:464] Registering TS with master...
I20260809 09:32:54.699272 30075 heartbeater.cc:511] Master 127.20.125.190:33605 requested a full tablet report, sending...
I20260809 09:32:54.700314 29414 ts_manager.cc:194] Registered new tserver with Master: 3a9fba53d35a4a848a139aee1bb2a7ce (127.20.125.131:43603)
I20260809 09:32:54.700927 29414 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.125.131@KRBTEST.COM'} at 127.20.125.131:46125
I20260809 09:32:54.706962 20982 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786267941944493-20982-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786267941944493-20982-0/krb5kdc/test-admin.keytab.
Aug 09 09:32:54 dist-test-slave-7kzw krb5kdc[29349](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786267974, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 09:32:54.726743 20982 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260809 09:32:54.729784 30090 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 09:32:54.728233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41840 (local address 127.20.125.129:36661)
0809 09:32:54.728584 (+ 351us) server_negotiation.cc:206] Beginning negotiation
0809 09:32:54.728591 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 09:32:54.728624 (+ 33us) server_negotiation.cc:407] Connection header received
0809 09:32:54.728674 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:32:54.728678 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:32:54.728873 (+ 195us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:32:54.729029 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:32:54.729602 (+ 573us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41840: BlockingRecv error: recv got EOF from 127.0.0.1:41840 (error 108)
Metrics: {"server-negotiator.queue_time_us":224,"thread_start_us":107,"threads_started":1}
W20260809 09:32:54.729885 20982 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786267941944493-20982-0/krb5kdc/krb5cc' of type FILE
Aug 09 09:32:54 dist-test-slave-7kzw krb5kdc[29349](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786267974, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 09:32:54.731115 20982 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 09 09:32:54 dist-test-slave-7kzw krb5kdc[29349](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786267974, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 09:32:54.732132 20982 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 09 09:32:54 dist-test-slave-7kzw krb5kdc[29349](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786267974, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.125.129@KRBTEST.COM
I20260809 09:32:54.742831 30090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:54.733054 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41848 (local address 127.20.125.129:36661)
0809 09:32:54.733208 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0809 09:32:54.733212 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 09:32:54.733226 (+ 14us) server_negotiation.cc:407] Connection header received
0809 09:32:54.733344 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:32:54.733346 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:32:54.733386 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:32:54.733456 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:32:54.734050 (+ 594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:54.735009 (+ 959us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:54.735891 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:54.736611 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:54.736775 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:54.739347 (+ 2572us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:32:54.739356 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:32:54.739370 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:32:54.739393 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:32:54.740996 (+ 1603us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:32:54.741456 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:32:54.741462 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:32:54.741463 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:32:54.741545 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:32:54.741828 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:32:54.741831 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:32:54.741833 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:32:54.742078 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:32:54.742319 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0809 09:32:54.742512 (+ 193us) server_negotiation.cc:299] Negotiation successful
0809 09:32:54.742703 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
W20260809 09:32:54.746676 30090 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 09:32:54.745574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41852 (local address 127.20.125.129:36661)
0809 09:32:54.745703 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0809 09:32:54.745707 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 09:32:54.745844 (+ 137us) server_negotiation.cc:407] Connection header received
0809 09:32:54.745985 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:32:54.745988 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:32:54.746031 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:32:54.746114 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:32:54.746569 (+ 455us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41852: BlockingRecv error: recv got EOF from 127.0.0.1:41852 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
W20260809 09:32:54.746774 20982 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786267941944493-20982-0/krb5kdc/krb5cc' of type FILE
Aug 09 09:32:54 dist-test-slave-7kzw krb5kdc[29349](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786267974, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 09:32:54.747926 20982 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 09 09:32:54 dist-test-slave-7kzw krb5kdc[29349](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786267974, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 09:32:54.748895 20982 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 09 09:32:54 dist-test-slave-7kzw krb5kdc[29349](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786267974, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.125.129@KRBTEST.COM
I20260809 09:32:54.757475 30090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:54.749749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41856 (local address 127.20.125.129:36661)
0809 09:32:54.749878 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0809 09:32:54.749882 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 09:32:54.749895 (+ 13us) server_negotiation.cc:407] Connection header received
0809 09:32:54.749997 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:32:54.749999 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:32:54.750162 (+ 163us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:32:54.750261 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:32:54.750864 (+ 603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:54.751351 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:54.752115 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:54.752342 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:54.752419 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:54.754569 (+ 2150us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:32:54.754572 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:32:54.754573 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:32:54.754593 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:32:54.755855 (+ 1262us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:32:54.756266 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:32:54.756269 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:32:54.756270 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:32:54.756352 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:32:54.756610 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:32:54.756614 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:32:54.756615 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:32:54.756832 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:32:54.756991 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0809 09:32:54.757159 (+ 168us) server_negotiation.cc:299] Negotiation successful
0809 09:32:54.757310 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
W20260809 09:32:54.760020 30090 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 09:32:54.759165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41862 (local address 127.20.125.129:36661)
0809 09:32:54.759289 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0809 09:32:54.759292 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 09:32:54.759320 (+ 28us) server_negotiation.cc:407] Connection header received
0809 09:32:54.759454 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:32:54.759456 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:32:54.759492 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:32:54.759556 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:32:54.759926 (+ 370us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41862: BlockingRecv error: recv got EOF from 127.0.0.1:41862 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
W20260809 09:32:54.760114 20982 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786267941944493-20982-0/krb5kdc/krb5cc' of type FILE
Aug 09 09:32:54 dist-test-slave-7kzw krb5kdc[29349](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786267974, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 09:32:54.761145 20982 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 09 09:32:54 dist-test-slave-7kzw krb5kdc[29349](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786267974, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 09:32:54.762044 20982 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 09 09:32:54 dist-test-slave-7kzw krb5kdc[29349](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786267974, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.125.129@KRBTEST.COM
I20260809 09:32:54.770598 30090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:54.762914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41870 (local address 127.20.125.129:36661)
0809 09:32:54.763032 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0809 09:32:54.763036 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 09:32:54.763138 (+ 102us) server_negotiation.cc:407] Connection header received
0809 09:32:54.763283 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:32:54.763285 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:32:54.763328 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:32:54.763409 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:32:54.763912 (+ 503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:54.764411 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:54.765195 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:54.765408 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:54.765513 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:54.767804 (+ 2291us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:32:54.767808 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:32:54.767809 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:32:54.767830 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:32:54.768977 (+ 1147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:32:54.769402 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:32:54.769410 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:32:54.769411 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:32:54.769483 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:32:54.769762 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:32:54.769767 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:32:54.769769 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:32:54.769923 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:32:54.770082 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0809 09:32:54.770314 (+ 232us) server_negotiation.cc:299] Negotiation successful
0809 09:32:54.770421 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260809 09:32:54.771590 20982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu with pid 29497
I20260809 09:32:54.786157 20982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu with pid 29683
I20260809 09:32:54.795485 20982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu with pid 29883
I20260809 09:32:54.812772 20982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu with pid 29368
2026-08-09T09:32:54Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (2490 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 09:32:54 dist-test-slave-7kzw krb5kdc[30113](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 09:32:54 dist-test-slave-7kzw krb5kdc[30113](info): set up 2 sockets
Aug 09 09:32:54 dist-test-slave-7kzw krb5kdc[30113](info): commencing operation
krb5kdc: starting...
W20260809 09:32:56.055977 20982 mini_kdc.cc:121] Time spent starting KDC: real 1.215s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/krb5kdc/test-user.keytab.
Aug 09 09:32:56 dist-test-slave-7kzw krb5kdc[30113](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786267976, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T09:32:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T09:32:56Z Disabled control of system clock
WARNING: no policy specified for oryx/127.20.125.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.20.125.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.20.125.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/krb5kdc/oryx_127.20.125.190.keytab.
Entry for principal oryx/127.20.125.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/krb5kdc/oryx_127.20.125.190.keytab.
I20260809 09:32:56.191942 20982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
/tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.125.190:37175
--webserver_interface=127.20.125.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/krb5kdc/oryx_127.20.125.190.keytab
--principal=oryx/127.20.125.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.125.148:36081
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.125.190:37175
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 09:32:56.309191 30133 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 09:32:56.309525 30133 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 09:32:56.309571 30133 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 09:32:56.313798 30133 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 09:32:56.313865 30133 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 09:32:56.313884 30133 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 09:32:56.313901 30133 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 09:32:56.313917 30133 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 09:32:56.319607 30133 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.125.148:36081
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.125.190:37175
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/krb5kdc/oryx_127.20.125.190.keytab
--principal=oryx/127.20.125.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.125.190:37175
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.125.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30133
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 08:38:55 UTC on ccycloud.quasar-njjleu.root.comops.site
build id 4575935
I20260809 09:32:56.320753 30133 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 09:32:56.321905 30133 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 09:32:56.331278 30140 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:32:56.331313 30141 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:32:56.331271 30145 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:32:56.331629 30133 server_base.cc:1034] running on GCE node
I20260809 09:32:56.332253 30133 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 09:32:56.333482 30133 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 09:32:56.334692 30133 hybrid_clock.cc:648] HybridClock initialized: now 1786267976334666 us; error 46 us; skew 500 ppm
Aug 09 09:32:56 dist-test-slave-7kzw krb5kdc[30113](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786267976, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.125.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 09:32:56.337579 30133 init.cc:377] Logged in from keytab as oryx/127.20.125.190@KRBTEST.COM (short username oryx)
I20260809 09:32:56.338655 30133 webserver.cc:460] Webserver started at http://127.20.125.190:46631/ using document root <none> and password file <none>
I20260809 09:32:56.339269 30133 fs_manager.cc:359] Metadata directory not provided
I20260809 09:32:56.339366 30133 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 09:32:56.339615 30133 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 09:32:56.341715 30133 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/minicluster-data/master-0/data/instance:
uuid: "ec1960de545a42ba9ac3e3e9adc99056"
format_stamp: "Formatted at 2026-08-09 09:32:56 on dist-test-slave-7kzw"
server_key: "20e572b984cfb295d046c43e31ea816e"
server_key_iv: "31a9224a8d702d20732dadf99bd68e18"
server_key_version: "encryptionkey@0"
I20260809 09:32:56.342306 30133 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/minicluster-data/master-0/wal/instance:
uuid: "ec1960de545a42ba9ac3e3e9adc99056"
format_stamp: "Formatted at 2026-08-09 09:32:56 on dist-test-slave-7kzw"
server_key: "20e572b984cfb295d046c43e31ea816e"
server_key_iv: "31a9224a8d702d20732dadf99bd68e18"
server_key_version: "encryptionkey@0"
I20260809 09:32:56.346163 30133 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.000s
I20260809 09:32:56.348722 30159 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 09:32:56.349982 30133 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 09:32:56.350150 30133 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/minicluster-data/master-0/data,/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/minicluster-data/master-0/wal
uuid: "ec1960de545a42ba9ac3e3e9adc99056"
format_stamp: "Formatted at 2026-08-09 09:32:56 on dist-test-slave-7kzw"
server_key: "20e572b984cfb295d046c43e31ea816e"
server_key_iv: "31a9224a8d702d20732dadf99bd68e18"
server_key_version: "encryptionkey@0"
I20260809 09:32:56.350270 30133 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 09:32:56.361449 30133 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 09:32:56.364836 30133 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 09:32:56.365057 30133 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 09:32:56.372802 30133 rpc_server.cc:307] RPC server started. Bound to: 127.20.125.190:37175
I20260809 09:32:56.373142 30236 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.125.190:37175 every 8 connection(s)
I20260809 09:32:56.373891 30133 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/minicluster-data/master-0/data/info.pb
I20260809 09:32:56.376935 30238 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 09:32:56.379693 20982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu as pid 30133
I20260809 09:32:56.379868 20982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/minicluster-data/master-0/wal/instance
I20260809 09:32:56.380184 20982 external_mini_cluster.cc:1428] Setting key 0acf5893aee598bffa6cee141bc0ab44
I20260809 09:32:56.383487 30238 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ec1960de545a42ba9ac3e3e9adc99056: Bootstrap starting.
I20260809 09:32:56.385895 30238 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ec1960de545a42ba9ac3e3e9adc99056: Neither blocks nor log segments found. Creating new log.
I20260809 09:32:56.386804 30238 log.cc:824] T 00000000000000000000000000000000 P ec1960de545a42ba9ac3e3e9adc99056: Log is configured to *not* fsync() on all Append() calls
I20260809 09:32:56.389446 30238 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ec1960de545a42ba9ac3e3e9adc99056: No bootstrap required, opened a new log
Aug 09 09:32:56 dist-test-slave-7kzw krb5kdc[30113](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786267976, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.20.125.190@KRBTEST.COM
I20260809 09:32:56.393167 30238 raft_consensus.cc:348] T 00000000000000000000000000000000 P ec1960de545a42ba9ac3e3e9adc99056 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec1960de545a42ba9ac3e3e9adc99056" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 37175 } }
I20260809 09:32:56.393443 30238 raft_consensus.cc:374] T 00000000000000000000000000000000 P ec1960de545a42ba9ac3e3e9adc99056 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 09:32:56.393538 30238 raft_consensus.cc:729] T 00000000000000000000000000000000 P ec1960de545a42ba9ac3e3e9adc99056 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec1960de545a42ba9ac3e3e9adc99056, State: Initialized, Role: FOLLOWER
I20260809 09:32:56.394065 30238 consensus_queue.cc:260] T 00000000000000000000000000000000 P ec1960de545a42ba9ac3e3e9adc99056 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec1960de545a42ba9ac3e3e9adc99056" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 37175 } }
I20260809 09:32:56.394238 30238 raft_consensus.cc:388] T 00000000000000000000000000000000 P ec1960de545a42ba9ac3e3e9adc99056 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 09:32:56.394337 30238 raft_consensus.cc:482] T 00000000000000000000000000000000 P ec1960de545a42ba9ac3e3e9adc99056 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 09:32:56.394448 30238 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ec1960de545a42ba9ac3e3e9adc99056 [term 0 FOLLOWER]: Advancing to term 1
I20260809 09:32:56.395445 30238 raft_consensus.cc:504] T 00000000000000000000000000000000 P ec1960de545a42ba9ac3e3e9adc99056 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec1960de545a42ba9ac3e3e9adc99056" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 37175 } }
I20260809 09:32:56.395864 30238 leader_election.cc:302] T 00000000000000000000000000000000 P ec1960de545a42ba9ac3e3e9adc99056 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ec1960de545a42ba9ac3e3e9adc99056; no voters:
I20260809 09:32:56.396229 30238 leader_election.cc:288] T 00000000000000000000000000000000 P ec1960de545a42ba9ac3e3e9adc99056 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 09:32:56.396332 30245 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ec1960de545a42ba9ac3e3e9adc99056 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 09:32:56.396549 30245 raft_consensus.cc:686] T 00000000000000000000000000000000 P ec1960de545a42ba9ac3e3e9adc99056 [term 1 LEADER]: Becoming Leader. State: Replica: ec1960de545a42ba9ac3e3e9adc99056, State: Running, Role: LEADER
I20260809 09:32:56.396848 30241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:56.381655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52626 (local address 127.20.125.190:37175)
0809 09:32:56.382154 (+ 499us) server_negotiation.cc:206] Beginning negotiation
0809 09:32:56.382170 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 09:32:56.382211 (+ 41us) server_negotiation.cc:407] Connection header received
0809 09:32:56.382990 (+ 779us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:32:56.383011 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:32:56.383368 (+ 357us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:32:56.383739 (+ 371us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:32:56.384658 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:56.385759 (+ 1101us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:56.386835 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:56.387567 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:56.387755 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:56.391024 (+ 3269us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:32:56.391031 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:32:56.391047 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:32:56.391077 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:32:56.393581 (+ 2504us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:32:56.394118 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:32:56.394124 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:32:56.394130 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:32:56.394286 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:32:56.394663 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:32:56.394666 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:32:56.394668 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:32:56.395070 (+ 402us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:32:56.395309 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0809 09:32:56.395564 (+ 255us) server_negotiation.cc:299] Negotiation successful
0809 09:32:56.395869 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":333,"thread_start_us":137,"threads_started":1}
I20260809 09:32:56.396919 30245 consensus_queue.cc:237] T 00000000000000000000000000000000 P ec1960de545a42ba9ac3e3e9adc99056 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec1960de545a42ba9ac3e3e9adc99056" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 37175 } }
I20260809 09:32:56.397614 30238 sys_catalog.cc:564] T 00000000000000000000000000000000 P ec1960de545a42ba9ac3e3e9adc99056 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 09:32:56.399842 30246 sys_catalog.cc:455] T 00000000000000000000000000000000 P ec1960de545a42ba9ac3e3e9adc99056 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ec1960de545a42ba9ac3e3e9adc99056" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec1960de545a42ba9ac3e3e9adc99056" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 37175 } } }
I20260809 09:32:56.399981 30246 sys_catalog.cc:458] T 00000000000000000000000000000000 P ec1960de545a42ba9ac3e3e9adc99056 [sys.catalog]: This master's current role is: LEADER
I20260809 09:32:56.400357 30265 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 09:32:56.400305 30247 sys_catalog.cc:455] T 00000000000000000000000000000000 P ec1960de545a42ba9ac3e3e9adc99056 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ec1960de545a42ba9ac3e3e9adc99056. Latest consensus state: current_term: 1 leader_uuid: "ec1960de545a42ba9ac3e3e9adc99056" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec1960de545a42ba9ac3e3e9adc99056" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 37175 } } }
I20260809 09:32:56.400842 30247 sys_catalog.cc:458] T 00000000000000000000000000000000 P ec1960de545a42ba9ac3e3e9adc99056 [sys.catalog]: This master's current role is: LEADER
I20260809 09:32:56.404179 30265 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 09:32:56.410840 30265 catalog_manager.cc:1347] Generated new cluster ID: a876011cffd24bdd84e8602f372dcb24
I20260809 09:32:56.410918 30265 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 09:32:56.430068 30265 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 09:32:56.431463 30265 catalog_manager.cc:1504] Loading token signing keys...
I20260809 09:32:56.443496 30265 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ec1960de545a42ba9ac3e3e9adc99056: Generated new TSK 0
I20260809 09:32:56.444377 30265 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.20.125.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.20.125.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.20.125.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/krb5kdc/oryx_127.20.125.129.keytab.
Entry for principal oryx/127.20.125.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/krb5kdc/oryx_127.20.125.129.keytab.
I20260809 09:32:56.481392 20982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
/tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.125.129:0
--local_ip_for_outbound_sockets=127.20.125.129
--webserver_interface=127.20.125.129
--webserver_port=0
--tserver_master_addrs=127.20.125.190:37175
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/krb5kdc/oryx_127.20.125.129.keytab
--principal=oryx/127.20.125.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.125.148:36081
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 09:32:56.595498 30275 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 09:32:56.595855 30275 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 09:32:56.595950 30275 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 09:32:56.600329 30275 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 09:32:56.600458 30275 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 09:32:56.600557 30275 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.125.129
I20260809 09:32:56.605968 30275 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.125.148:36081
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/krb5kdc/oryx_127.20.125.129.keytab
--principal=oryx/127.20.125.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.125.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.125.129
--webserver_port=0
--tserver_master_addrs=127.20.125.190:37175
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30275
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.125.129
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 08:38:55 UTC on ccycloud.quasar-njjleu.root.comops.site
build id 4575935
I20260809 09:32:56.607070 30275 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 09:32:56.608142 30275 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 09:32:56.618391 30284 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:32:56.618372 30289 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:32:56.618584 30275 server_base.cc:1034] running on GCE node
W20260809 09:32:56.618397 30285 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:32:56.619215 30275 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 09:32:56.619894 30275 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 09:32:56.621120 30275 hybrid_clock.cc:648] HybridClock initialized: now 1786267976621102 us; error 37 us; skew 500 ppm
Aug 09 09:32:56 dist-test-slave-7kzw krb5kdc[30113](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786267976, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.125.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 09:32:56.627250 30275 init.cc:377] Logged in from keytab as oryx/127.20.125.129@KRBTEST.COM (short username oryx)
I20260809 09:32:56.628258 30275 webserver.cc:460] Webserver started at http://127.20.125.129:35525/ using document root <none> and password file <none>
I20260809 09:32:56.628844 30275 fs_manager.cc:359] Metadata directory not provided
I20260809 09:32:56.628943 30275 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 09:32:56.629184 30275 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 09:32:56.631446 30275 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/minicluster-data/ts-0/data/instance:
uuid: "143f334e51d54d17911a2d2ebb0b6554"
format_stamp: "Formatted at 2026-08-09 09:32:56 on dist-test-slave-7kzw"
server_key: "1e2e593bc163a00e9387565016b42f77"
server_key_iv: "4ccac51aca026c965bfaae05b91b9a72"
server_key_version: "encryptionkey@0"
I20260809 09:32:56.632050 30275 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/minicluster-data/ts-0/wal/instance:
uuid: "143f334e51d54d17911a2d2ebb0b6554"
format_stamp: "Formatted at 2026-08-09 09:32:56 on dist-test-slave-7kzw"
server_key: "1e2e593bc163a00e9387565016b42f77"
server_key_iv: "4ccac51aca026c965bfaae05b91b9a72"
server_key_version: "encryptionkey@0"
I20260809 09:32:56.635987 30275 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260809 09:32:56.638602 30297 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 09:32:56.639989 30275 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 09:32:56.640177 30275 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/minicluster-data/ts-0/data,/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/minicluster-data/ts-0/wal
uuid: "143f334e51d54d17911a2d2ebb0b6554"
format_stamp: "Formatted at 2026-08-09 09:32:56 on dist-test-slave-7kzw"
server_key: "1e2e593bc163a00e9387565016b42f77"
server_key_iv: "4ccac51aca026c965bfaae05b91b9a72"
server_key_version: "encryptionkey@0"
I20260809 09:32:56.640318 30275 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 09:32:56.653170 30275 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 09:32:56.656675 30275 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 09:32:56.656837 30275 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 09:32:56.657404 30275 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 09:32:56.658270 30275 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 09:32:56.658320 30275 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 09:32:56.658368 30275 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 09:32:56.658385 30275 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 09:32:56.667524 30275 rpc_server.cc:307] RPC server started. Bound to: 127.20.125.129:42545
I20260809 09:32:56.667554 30480 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.125.129:42545 every 8 connection(s)
I20260809 09:32:56.668404 30275 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/minicluster-data/ts-0/data/info.pb
I20260809 09:32:56.668951 20982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu as pid 30275
I20260809 09:32:56.669071 20982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/minicluster-data/ts-0/wal/instance
I20260809 09:32:56.669425 20982 external_mini_cluster.cc:1428] Setting key 34047311eb498a24b9ad7c7a3c9e055d
Aug 09 09:32:56 dist-test-slave-7kzw krb5kdc[30113](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786267976, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.125.129@KRBTEST.COM for oryx/127.20.125.190@KRBTEST.COM
WARNING: no policy specified for oryx/127.20.125.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.20.125.130@KRBTEST.COM" created.
I20260809 09:32:56.684840 30241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:56.670390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.129:45951 (local address 127.20.125.190:37175)
0809 09:32:56.670544 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0809 09:32:56.670548 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 09:32:56.671720 (+ 1172us) server_negotiation.cc:407] Connection header received
0809 09:32:56.673792 (+ 2072us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:32:56.673795 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:32:56.673853 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:32:56.673935 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:32:56.675236 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:56.675789 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:56.677058 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:56.677303 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:56.677421 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:56.680656 (+ 3235us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:32:56.680664 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:32:56.680669 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:32:56.680704 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:32:56.682438 (+ 1734us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:32:56.683052 (+ 614us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:32:56.683059 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:32:56.683062 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:32:56.683165 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:32:56.683585 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:32:56.683589 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:32:56.683591 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:32:56.683790 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:32:56.683912 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0809 09:32:56.684484 (+ 572us) server_negotiation.cc:299] Negotiation successful
0809 09:32:56.684640 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260809 09:32:56.685937 30486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:56.670994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:37175 (local address 127.20.125.129:45951)
0809 09:32:56.671539 (+ 545us) negotiation.cc:107] Waiting for socket to connect
0809 09:32:56.671600 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0809 09:32:56.673564 (+ 1964us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:32:56.674111 (+ 547us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:32:56.674118 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:32:56.674566 (+ 448us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:32:56.675076 (+ 510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:32:56.675090 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:32:56.675935 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:32:56.675939 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:32:56.676920 (+ 981us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:32:56.676931 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:32:56.677468 (+ 537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:32:56.677473 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:32:56.677725 (+ 252us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:56.678453 (+ 728us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 09:32:56.678502 (+ 49us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:32:56.680456 (+ 1954us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 09:32:56.682605 (+ 2149us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:32:56.682611 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:32:56.682625 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:32:56.682912 (+ 287us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:32:56.683287 (+ 375us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:32:56.683296 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:32:56.683297 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:32:56.683447 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:32:56.683911 (+ 464us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 09:32:56.683919 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 09:32:56.684199 (+ 280us) client_negotiation.cc:769] Sending connection context
0809 09:32:56.684410 (+ 211us) client_negotiation.cc:240] Negotiation successful
0809 09:32:56.684782 (+ 372us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":308,"thread_start_us":133,"threads_started":1}
I20260809 09:32:56.687553 30481 heartbeater.cc:347] Connected to a master server at 127.20.125.190:37175
I20260809 09:32:56.687839 30481 heartbeater.cc:464] Registering TS with master...
I20260809 09:32:56.688382 30481 heartbeater.cc:511] Master 127.20.125.190:37175 requested a full tablet report, sending...
I20260809 09:32:56.690176 30182 ts_manager.cc:194] Registered new tserver with Master: 143f334e51d54d17911a2d2ebb0b6554 (127.20.125.129:42545)
I20260809 09:32:56.691505 30182 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.20.125.129@KRBTEST.COM'} at 127.20.125.129:45951
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.20.125.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/krb5kdc/oryx_127.20.125.130.keytab.
Entry for principal oryx/127.20.125.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/krb5kdc/oryx_127.20.125.130.keytab.
I20260809 09:32:56.701545 20982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
/tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.125.130:0
--local_ip_for_outbound_sockets=127.20.125.130
--webserver_interface=127.20.125.130
--webserver_port=0
--tserver_master_addrs=127.20.125.190:37175
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/krb5kdc/oryx_127.20.125.130.keytab
--principal=oryx/127.20.125.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.125.148:36081
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 09:32:56.820472 30491 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 09:32:56.820737 30491 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 09:32:56.820823 30491 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 09:32:56.824875 30491 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 09:32:56.824937 30491 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 09:32:56.825028 30491 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.125.130
I20260809 09:32:56.830224 30491 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.125.148:36081
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/krb5kdc/oryx_127.20.125.130.keytab
--principal=oryx/127.20.125.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.125.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.125.130
--webserver_port=0
--tserver_master_addrs=127.20.125.190:37175
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30491
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.125.130
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 08:38:55 UTC on ccycloud.quasar-njjleu.root.comops.site
build id 4575935
I20260809 09:32:56.831296 30491 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 09:32:56.832311 30491 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 09:32:56.842724 30506 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:32:56.842670 30498 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:32:56.842824 30491 server_base.cc:1034] running on GCE node
W20260809 09:32:56.842963 30502 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:32:56.843436 30491 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 09:32:56.844069 30491 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 09:32:56.845260 30491 hybrid_clock.cc:648] HybridClock initialized: now 1786267976845241 us; error 59 us; skew 500 ppm
Aug 09 09:32:56 dist-test-slave-7kzw krb5kdc[30113](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786267976, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.125.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 09:32:56.848065 30491 init.cc:377] Logged in from keytab as oryx/127.20.125.130@KRBTEST.COM (short username oryx)
I20260809 09:32:56.849004 30491 webserver.cc:460] Webserver started at http://127.20.125.130:33213/ using document root <none> and password file <none>
I20260809 09:32:56.849579 30491 fs_manager.cc:359] Metadata directory not provided
I20260809 09:32:56.849674 30491 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 09:32:56.849922 30491 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 09:32:56.852038 30491 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/minicluster-data/ts-1/data/instance:
uuid: "8bbe397caa0242579d9743094319a762"
format_stamp: "Formatted at 2026-08-09 09:32:56 on dist-test-slave-7kzw"
server_key: "9f42e2e53fdd35eef20409db80a49ef8"
server_key_iv: "c0601ac9cb68cab291ed792dd1cf81e3"
server_key_version: "encryptionkey@0"
I20260809 09:32:56.852639 30491 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/minicluster-data/ts-1/wal/instance:
uuid: "8bbe397caa0242579d9743094319a762"
format_stamp: "Formatted at 2026-08-09 09:32:56 on dist-test-slave-7kzw"
server_key: "9f42e2e53fdd35eef20409db80a49ef8"
server_key_iv: "c0601ac9cb68cab291ed792dd1cf81e3"
server_key_version: "encryptionkey@0"
I20260809 09:32:56.856515 30491 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.001s
I20260809 09:32:56.858944 30514 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 09:32:56.860136 30491 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 09:32:56.860286 30491 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/minicluster-data/ts-1/data,/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/minicluster-data/ts-1/wal
uuid: "8bbe397caa0242579d9743094319a762"
format_stamp: "Formatted at 2026-08-09 09:32:56 on dist-test-slave-7kzw"
server_key: "9f42e2e53fdd35eef20409db80a49ef8"
server_key_iv: "c0601ac9cb68cab291ed792dd1cf81e3"
server_key_version: "encryptionkey@0"
I20260809 09:32:56.860401 30491 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 09:32:56.872339 30491 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 09:32:56.875720 30491 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 09:32:56.875946 30491 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 09:32:56.876586 30491 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 09:32:56.877499 30491 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 09:32:56.877578 30491 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 09:32:56.877669 30491 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 09:32:56.877729 30491 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 09:32:56.888062 30491 rpc_server.cc:307] RPC server started. Bound to: 127.20.125.130:44297
I20260809 09:32:56.888090 30686 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.125.130:44297 every 8 connection(s)
I20260809 09:32:56.889110 30491 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/minicluster-data/ts-1/data/info.pb
I20260809 09:32:56.899369 20982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu as pid 30491
I20260809 09:32:56.899531 20982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/minicluster-data/ts-1/wal/instance
Aug 09 09:32:56 dist-test-slave-7kzw krb5kdc[30113](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786267976, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.125.130@KRBTEST.COM for oryx/127.20.125.190@KRBTEST.COM
I20260809 09:32:56.899914 20982 external_mini_cluster.cc:1428] Setting key b568c8cf15f71fc4d82e23f1aa8eb4d2
I20260809 09:32:56.905016 30241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:56.891143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.130:41753 (local address 127.20.125.190:37175)
0809 09:32:56.891333 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0809 09:32:56.891337 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 09:32:56.892189 (+ 852us) server_negotiation.cc:407] Connection header received
0809 09:32:56.893596 (+ 1407us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:32:56.893599 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:32:56.893660 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:32:56.893781 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:32:56.895069 (+ 1288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:56.895603 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:56.896949 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:56.897177 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:56.897286 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:56.900479 (+ 3193us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:32:56.900484 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:32:56.900486 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:32:56.900521 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:32:56.902352 (+ 1831us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:32:56.903045 (+ 693us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:32:56.903048 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:32:56.903050 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:32:56.903140 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:32:56.903661 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:32:56.903664 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:32:56.903665 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:32:56.903979 (+ 314us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:32:56.904102 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0809 09:32:56.904616 (+ 514us) server_negotiation.cc:299] Negotiation successful
0809 09:32:56.904779 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260809 09:32:56.905721 30691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:56.891562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:37175 (local address 127.20.125.130:41753)
0809 09:32:56.892039 (+ 477us) negotiation.cc:107] Waiting for socket to connect
0809 09:32:56.892082 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 09:32:56.893376 (+ 1294us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:32:56.893957 (+ 581us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:32:56.893964 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:32:56.894329 (+ 365us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:32:56.894920 (+ 591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:32:56.894934 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:32:56.895739 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:32:56.895745 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:32:56.896811 (+ 1066us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:32:56.896821 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:32:56.897337 (+ 516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:32:56.897341 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:32:56.897504 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:56.898205 (+ 701us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 09:32:56.898222 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:32:56.900288 (+ 2066us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 09:32:56.902503 (+ 2215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:32:56.902512 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:32:56.902532 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:32:56.902928 (+ 396us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:32:56.903261 (+ 333us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:32:56.903267 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:32:56.903272 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:32:56.903502 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:32:56.904106 (+ 604us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 09:32:56.904111 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 09:32:56.904348 (+ 237us) client_negotiation.cc:769] Sending connection context
0809 09:32:56.904500 (+ 152us) client_negotiation.cc:240] Negotiation successful
0809 09:32:56.904750 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":286,"thread_start_us":133,"threads_started":1}
I20260809 09:32:56.907122 30687 heartbeater.cc:347] Connected to a master server at 127.20.125.190:37175
I20260809 09:32:56.907423 30687 heartbeater.cc:464] Registering TS with master...
I20260809 09:32:56.908030 30687 heartbeater.cc:511] Master 127.20.125.190:37175 requested a full tablet report, sending...
I20260809 09:32:56.909255 30182 ts_manager.cc:194] Registered new tserver with Master: 8bbe397caa0242579d9743094319a762 (127.20.125.130:44297)
I20260809 09:32:56.909876 30182 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.20.125.130@KRBTEST.COM'} at 127.20.125.130:41753
WARNING: no policy specified for oryx/127.20.125.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.20.125.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.20.125.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/krb5kdc/oryx_127.20.125.131.keytab.
Entry for principal oryx/127.20.125.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/krb5kdc/oryx_127.20.125.131.keytab.
I20260809 09:32:56.933982 20982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
/tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.125.131:0
--local_ip_for_outbound_sockets=127.20.125.131
--webserver_interface=127.20.125.131
--webserver_port=0
--tserver_master_addrs=127.20.125.190:37175
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/krb5kdc/oryx_127.20.125.131.keytab
--principal=oryx/127.20.125.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.125.148:36081
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 09:32:57.054039 30697 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 09:32:57.054319 30697 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 09:32:57.054402 30697 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 09:32:57.058609 30697 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 09:32:57.058665 30697 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 09:32:57.058722 30697 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.125.131
I20260809 09:32:57.063867 30697 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.125.148:36081
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/krb5kdc/oryx_127.20.125.131.keytab
--principal=oryx/127.20.125.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.125.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.125.131
--webserver_port=0
--tserver_master_addrs=127.20.125.190:37175
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30697
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.125.131
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 08:38:55 UTC on ccycloud.quasar-njjleu.root.comops.site
build id 4575935
I20260809 09:32:57.064841 30697 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 09:32:57.065867 30697 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 09:32:57.075683 30707 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:32:57.075798 30705 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:32:57.075716 30709 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:32:57.076040 30697 server_base.cc:1034] running on GCE node
I20260809 09:32:57.076637 30697 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 09:32:57.077387 30697 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 09:32:57.078553 30697 hybrid_clock.cc:648] HybridClock initialized: now 1786267977078517 us; error 45 us; skew 500 ppm
Aug 09 09:32:57 dist-test-slave-7kzw krb5kdc[30113](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786267977, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.125.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 09:32:57.081629 30697 init.cc:377] Logged in from keytab as oryx/127.20.125.131@KRBTEST.COM (short username oryx)
I20260809 09:32:57.082646 30697 webserver.cc:460] Webserver started at http://127.20.125.131:38165/ using document root <none> and password file <none>
I20260809 09:32:57.083266 30697 fs_manager.cc:359] Metadata directory not provided
I20260809 09:32:57.083369 30697 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 09:32:57.083633 30697 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 09:32:57.085896 30697 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/minicluster-data/ts-2/data/instance:
uuid: "825ed73d552f4f8cbce972a3dc70ccd8"
format_stamp: "Formatted at 2026-08-09 09:32:57 on dist-test-slave-7kzw"
server_key: "0b550ab544b5e99f67e8b7905baf5fc7"
server_key_iv: "24adfb2f986e24968c618f4dc82821d2"
server_key_version: "encryptionkey@0"
I20260809 09:32:57.086597 30697 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/minicluster-data/ts-2/wal/instance:
uuid: "825ed73d552f4f8cbce972a3dc70ccd8"
format_stamp: "Formatted at 2026-08-09 09:32:57 on dist-test-slave-7kzw"
server_key: "0b550ab544b5e99f67e8b7905baf5fc7"
server_key_iv: "24adfb2f986e24968c618f4dc82821d2"
server_key_version: "encryptionkey@0"
I20260809 09:32:57.090797 30697 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.003s
I20260809 09:32:57.093465 30716 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 09:32:57.094858 30697 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 09:32:57.095005 30697 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/minicluster-data/ts-2/data,/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/minicluster-data/ts-2/wal
uuid: "825ed73d552f4f8cbce972a3dc70ccd8"
format_stamp: "Formatted at 2026-08-09 09:32:57 on dist-test-slave-7kzw"
server_key: "0b550ab544b5e99f67e8b7905baf5fc7"
server_key_iv: "24adfb2f986e24968c618f4dc82821d2"
server_key_version: "encryptionkey@0"
I20260809 09:32:57.095125 30697 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 09:32:57.130595 30697 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 09:32:57.134325 30697 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 09:32:57.134582 30697 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 09:32:57.135245 30697 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 09:32:57.136161 30697 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 09:32:57.136255 30697 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 09:32:57.136361 30697 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 09:32:57.136420 30697 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 09:32:57.146456 30697 rpc_server.cc:307] RPC server started. Bound to: 127.20.125.131:44279
I20260809 09:32:57.146528 30885 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.125.131:44279 every 8 connection(s)
I20260809 09:32:57.147547 30697 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/minicluster-data/ts-2/data/info.pb
I20260809 09:32:57.151867 20982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu as pid 30697
I20260809 09:32:57.151999 20982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/minicluster-data/ts-2/wal/instance
I20260809 09:32:57.152383 20982 external_mini_cluster.cc:1428] Setting key 217f209f6e9fc3b54dc29dba718575ed
Aug 09 09:32:57 dist-test-slave-7kzw krb5kdc[30113](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786267977, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.125.131@KRBTEST.COM for oryx/127.20.125.190@KRBTEST.COM
I20260809 09:32:57.162175 30241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:57.149428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.131:48449 (local address 127.20.125.190:37175)
0809 09:32:57.149583 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0809 09:32:57.149588 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 09:32:57.150412 (+ 824us) server_negotiation.cc:407] Connection header received
0809 09:32:57.152067 (+ 1655us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:32:57.152071 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:32:57.152133 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:32:57.152237 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:32:57.153671 (+ 1434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:57.154429 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:57.155820 (+ 1391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:57.156046 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:57.156168 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:57.158776 (+ 2608us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:32:57.158779 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:32:57.158781 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:32:57.158805 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:32:57.160160 (+ 1355us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:32:57.160723 (+ 563us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:32:57.160726 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:32:57.160727 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:32:57.160825 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:32:57.161176 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:32:57.161179 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:32:57.161180 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:32:57.161363 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:32:57.161446 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0809 09:32:57.161952 (+ 506us) server_negotiation.cc:299] Negotiation successful
0809 09:32:57.162047 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260809 09:32:57.163105 30892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:57.149806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:37175 (local address 127.20.125.131:48449)
0809 09:32:57.150260 (+ 454us) negotiation.cc:107] Waiting for socket to connect
0809 09:32:57.150304 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 09:32:57.151802 (+ 1498us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:32:57.152400 (+ 598us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:32:57.152407 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:32:57.152867 (+ 460us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:32:57.153533 (+ 666us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:32:57.153549 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:32:57.154582 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:32:57.154585 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:32:57.155696 (+ 1111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:32:57.155707 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:32:57.156277 (+ 570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:32:57.156285 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:32:57.156435 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:57.157053 (+ 618us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 09:32:57.157069 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:32:57.158623 (+ 1554us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 09:32:57.160295 (+ 1672us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:32:57.160301 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:32:57.160312 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:32:57.160586 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:32:57.160923 (+ 337us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:32:57.160925 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:32:57.160927 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:32:57.161067 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:32:57.161462 (+ 395us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 09:32:57.161467 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 09:32:57.161733 (+ 266us) client_negotiation.cc:769] Sending connection context
0809 09:32:57.161902 (+ 169us) client_negotiation.cc:240] Negotiation successful
0809 09:32:57.162154 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":256,"thread_start_us":113,"threads_started":1}
I20260809 09:32:57.164320 30888 heartbeater.cc:347] Connected to a master server at 127.20.125.190:37175
I20260809 09:32:57.164564 30888 heartbeater.cc:464] Registering TS with master...
I20260809 09:32:57.165197 30888 heartbeater.cc:511] Master 127.20.125.190:37175 requested a full tablet report, sending...
I20260809 09:32:57.166450 30181 ts_manager.cc:194] Registered new tserver with Master: 825ed73d552f4f8cbce972a3dc70ccd8 (127.20.125.131:44279)
I20260809 09:32:57.167092 30181 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.20.125.131@KRBTEST.COM'} at 127.20.125.131:48449
I20260809 09:32:57.178180 20982 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.20.125.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.20.125.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.20.125.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/krb5kdc/oryx_127.20.125.189.keytab.
Entry for principal oryx/127.20.125.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/krb5kdc/oryx_127.20.125.189.keytab.
I20260809 09:32:57.210764 20982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
/tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.125.189:44285
--webserver_interface=127.20.125.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/krb5kdc/oryx_127.20.125.189.keytab
--principal=oryx/127.20.125.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.125.148:36081
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.125.190:37175,127.20.125.189:44285
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 09:32:57.325815 30901 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 09:32:57.326076 30901 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 09:32:57.326114 30901 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 09:32:57.330375 30901 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 09:32:57.330442 30901 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 09:32:57.330510 30901 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 09:32:57.330543 30901 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 09:32:57.330583 30901 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 09:32:57.335902 30901 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.125.148:36081
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.20.125.190:37175,127.20.125.189:44285
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/krb5kdc/oryx_127.20.125.189.keytab
--principal=oryx/127.20.125.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.125.189:44285
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.20.125.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30901
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 08:38:55 UTC on ccycloud.quasar-njjleu.root.comops.site
build id 4575935
W20260809 09:32:57.336140 30901 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.20.125.190:37175,127.20.125.189:44285'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260809 09:32:57.337006 30901 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 09:32:57.338099 30901 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 09:32:57.346609 30913 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:32:57.346668 30910 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:32:57.346616 30909 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:32:57.346967 30901 server_base.cc:1034] running on GCE node
I20260809 09:32:57.347461 30901 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 09:32:57.348592 30901 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 09:32:57.349788 30901 hybrid_clock.cc:648] HybridClock initialized: now 1786267977349778 us; error 32 us; skew 500 ppm
Aug 09 09:32:57 dist-test-slave-7kzw krb5kdc[30113](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786267977, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.125.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 09:32:57.352741 30901 init.cc:377] Logged in from keytab as oryx/127.20.125.189@KRBTEST.COM (short username oryx)
I20260809 09:32:57.353652 30901 webserver.cc:460] Webserver started at http://127.20.125.189:34549/ using document root <none> and password file <none>
I20260809 09:32:57.354300 30901 fs_manager.cc:359] Metadata directory not provided
I20260809 09:32:57.354391 30901 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 09:32:57.354648 30901 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 09:32:57.356767 30901 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/minicluster-data/master-1/data/instance:
uuid: "3c029f42802647dea9a4c2f436be31eb"
format_stamp: "Formatted at 2026-08-09 09:32:57 on dist-test-slave-7kzw"
server_key: "fb7011462a080d57d7b61308f2cdf82a"
server_key_iv: "75126c7a7cdbf0258f06edb43de58aa7"
server_key_version: "encryptionkey@0"
I20260809 09:32:57.357308 30901 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/minicluster-data/master-1/wal/instance:
uuid: "3c029f42802647dea9a4c2f436be31eb"
format_stamp: "Formatted at 2026-08-09 09:32:57 on dist-test-slave-7kzw"
server_key: "fb7011462a080d57d7b61308f2cdf82a"
server_key_iv: "75126c7a7cdbf0258f06edb43de58aa7"
server_key_version: "encryptionkey@0"
I20260809 09:32:57.361176 30901 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 09:32:57.363631 30921 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 09:32:57.365270 30901 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 09:32:57.365409 30901 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/minicluster-data/master-1/data,/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/minicluster-data/master-1/wal
uuid: "3c029f42802647dea9a4c2f436be31eb"
format_stamp: "Formatted at 2026-08-09 09:32:57 on dist-test-slave-7kzw"
server_key: "fb7011462a080d57d7b61308f2cdf82a"
server_key_iv: "75126c7a7cdbf0258f06edb43de58aa7"
server_key_version: "encryptionkey@0"
I20260809 09:32:57.365523 30901 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 09:32:57.391433 30901 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 09:32:57.394965 30901 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 09:32:57.395193 30901 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 09:32:57.402532 30901 rpc_server.cc:307] RPC server started. Bound to: 127.20.125.189:44285
I20260809 09:32:57.402858 30993 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.125.189:44285 every 8 connection(s)
I20260809 09:32:57.403579 30901 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/minicluster-data/master-1/data/info.pb
I20260809 09:32:57.406566 30994 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 09:32:57.408720 20982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu as pid 30901
I20260809 09:32:57.410617 30994 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.125.190" port: 37175 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 09:32:57 dist-test-slave-7kzw krb5kdc[30113](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786267977, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.125.189@KRBTEST.COM for oryx/127.20.125.190@KRBTEST.COM
Aug 09 09:32:57 dist-test-slave-7kzw krb5kdc[30113](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786267976, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.20.125.189@KRBTEST.COM
I20260809 09:32:57.422878 30241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:57.412350 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52638 (local address 127.20.125.190:37175)
0809 09:32:57.412505 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0809 09:32:57.412509 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 09:32:57.413126 (+ 617us) server_negotiation.cc:407] Connection header received
0809 09:32:57.413292 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:32:57.413297 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:32:57.413346 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:32:57.413440 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:32:57.414121 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:57.414687 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:57.415588 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:57.415811 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:57.415926 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:57.418745 (+ 2819us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:32:57.418749 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:32:57.418755 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:32:57.418790 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:32:57.420487 (+ 1697us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:32:57.421145 (+ 658us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:32:57.421149 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:32:57.421150 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:32:57.421260 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:32:57.421665 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:32:57.421668 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:32:57.421669 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:32:57.421888 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:32:57.422022 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0809 09:32:57.422582 (+ 560us) server_negotiation.cc:299] Negotiation successful
0809 09:32:57.422706 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260809 09:32:57.423780 31001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:57.412303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:37175 (local address 127.0.0.1:52638)
0809 09:32:57.413002 (+ 699us) negotiation.cc:107] Waiting for socket to connect
0809 09:32:57.413039 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 09:32:57.413191 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:32:57.413491 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:32:57.413496 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:32:57.413837 (+ 341us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:32:57.413986 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:32:57.413995 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:32:57.414821 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:32:57.414824 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:32:57.415467 (+ 643us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:32:57.415475 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:32:57.415965 (+ 490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:32:57.415969 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:32:57.416110 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:57.416651 (+ 541us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 09:32:57.416676 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:32:57.418595 (+ 1919us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 09:32:57.420593 (+ 1998us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:32:57.420600 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:32:57.420610 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:32:57.421006 (+ 396us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:32:57.421355 (+ 349us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:32:57.421358 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:32:57.421360 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:32:57.421564 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:32:57.422036 (+ 472us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 09:32:57.422045 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 09:32:57.422308 (+ 263us) client_negotiation.cc:769] Sending connection context
0809 09:32:57.422537 (+ 229us) client_negotiation.cc:240] Negotiation successful
0809 09:32:57.422908 (+ 371us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":578,"mutex_wait_us":3,"thread_start_us":92,"threads_started":1}
I20260809 09:32:57.423784 30997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:57.410275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49560 (local address 127.20.125.189:44285)
0809 09:32:57.410708 (+ 433us) server_negotiation.cc:206] Beginning negotiation
0809 09:32:57.410723 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 09:32:57.410760 (+ 37us) server_negotiation.cc:407] Connection header received
0809 09:32:57.411465 (+ 705us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:32:57.411484 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:32:57.411807 (+ 323us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:32:57.412127 (+ 320us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:32:57.413026 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:57.414000 (+ 974us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:57.414944 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:57.415483 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:57.416116 (+ 633us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:57.419257 (+ 3141us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:32:57.419264 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:32:57.419276 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:32:57.419306 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:32:57.421144 (+ 1838us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:32:57.421763 (+ 619us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:32:57.421768 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:32:57.421769 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:32:57.421838 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:32:57.422129 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:32:57.422134 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:32:57.422135 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:32:57.422372 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:32:57.422607 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0809 09:32:57.422796 (+ 189us) server_negotiation.cc:299] Negotiation successful
0809 09:32:57.422973 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":84,"server-negotiator.queue_time_us":293,"spinlock_wait_cycles":20352,"thread_start_us":145,"threads_started":1}
I20260809 09:32:57.425866 30994 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.125.189" port: 44285 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 09:32:57 dist-test-slave-7kzw krb5kdc[30113](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786267977, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.125.189@KRBTEST.COM for oryx/127.20.125.189@KRBTEST.COM
I20260809 09:32:57.433657 30997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:57.426509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49566 (local address 127.20.125.189:44285)
0809 09:32:57.426618 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0809 09:32:57.426622 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 09:32:57.426643 (+ 21us) server_negotiation.cc:407] Connection header received
0809 09:32:57.426750 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:32:57.426752 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:32:57.426785 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:32:57.426864 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:32:57.427197 (+ 333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:57.427691 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:57.428385 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:57.428592 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:57.428675 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:57.430650 (+ 1975us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:32:57.430653 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:32:57.430655 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:32:57.430676 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:32:57.432097 (+ 1421us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:32:57.432470 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:32:57.432472 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:32:57.432473 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:32:57.432545 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:32:57.432778 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:32:57.432779 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:32:57.432780 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:32:57.432933 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:32:57.433008 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0809 09:32:57.433288 (+ 280us) server_negotiation.cc:299] Negotiation successful
0809 09:32:57.433376 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260809 09:32:57.433681 31001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:57.426417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.189:44285 (local address 127.0.0.1:49566)
0809 09:32:57.426568 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0809 09:32:57.426586 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 09:32:57.426645 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:32:57.426879 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:32:57.426881 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:32:57.427014 (+ 133us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:32:57.427101 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:32:57.427106 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:32:57.427786 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:32:57.427788 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:32:57.428294 (+ 506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:32:57.428301 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:32:57.428720 (+ 419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:32:57.428723 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:32:57.428809 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:57.429232 (+ 423us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 09:32:57.429243 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:32:57.430527 (+ 1284us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 09:32:57.432202 (+ 1675us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:32:57.432204 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:32:57.432206 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:32:57.432388 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:32:57.432623 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:32:57.432625 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:32:57.432626 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:32:57.432706 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:32:57.433043 (+ 337us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 09:32:57.433046 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 09:32:57.433124 (+ 78us) client_negotiation.cc:769] Sending connection context
0809 09:32:57.433221 (+ 97us) client_negotiation.cc:240] Negotiation successful
0809 09:32:57.433369 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":78}
I20260809 09:32:57.437661 30994 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3c029f42802647dea9a4c2f436be31eb: Bootstrap starting.
I20260809 09:32:57.440238 30994 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3c029f42802647dea9a4c2f436be31eb: Neither blocks nor log segments found. Creating new log.
I20260809 09:32:57.440994 30994 log.cc:824] T 00000000000000000000000000000000 P 3c029f42802647dea9a4c2f436be31eb: Log is configured to *not* fsync() on all Append() calls
I20260809 09:32:57.443265 30994 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3c029f42802647dea9a4c2f436be31eb: No bootstrap required, opened a new log
I20260809 09:32:57.446794 30994 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3c029f42802647dea9a4c2f436be31eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec1960de545a42ba9ac3e3e9adc99056" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 37175 } } peers { permanent_uuid: "3c029f42802647dea9a4c2f436be31eb" member_type: VOTER last_known_addr { host: "127.20.125.189" port: 44285 } }
I20260809 09:32:57.447032 30994 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3c029f42802647dea9a4c2f436be31eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 09:32:57.447124 30994 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3c029f42802647dea9a4c2f436be31eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c029f42802647dea9a4c2f436be31eb, State: Initialized, Role: FOLLOWER
I20260809 09:32:57.447597 30994 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3c029f42802647dea9a4c2f436be31eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec1960de545a42ba9ac3e3e9adc99056" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 37175 } } peers { permanent_uuid: "3c029f42802647dea9a4c2f436be31eb" member_type: VOTER last_known_addr { host: "127.20.125.189" port: 44285 } }
I20260809 09:32:57.448117 31003 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3c029f42802647dea9a4c2f436be31eb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec1960de545a42ba9ac3e3e9adc99056" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 37175 } } peers { permanent_uuid: "3c029f42802647dea9a4c2f436be31eb" member_type: VOTER last_known_addr { host: "127.20.125.189" port: 44285 } } }
I20260809 09:32:57.448239 31003 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3c029f42802647dea9a4c2f436be31eb [sys.catalog]: This master's current role is: FOLLOWER
I20260809 09:32:57.448854 30994 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3c029f42802647dea9a4c2f436be31eb [sys.catalog]: configured and running, proceeding with master startup.
W20260809 09:32:57.453850 31022 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3c029f42802647dea9a4c2f436be31eb: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 09:32:57.453951 31022 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 09:32:57.456214 30901 master_runner.cc:428] Detected that this master 3c029f42802647dea9a4c2f436be31eb is joining an existing cluster
I20260809 09:32:57.456269 30901 master_runner.cc:432] Initiating AddMaster RPC to add 127.20.125.189:44285
I20260809 09:32:57.467223 31031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:57.458457 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.189:44285 (local address 127.0.0.1:49568)
0809 09:32:57.459041 (+ 584us) negotiation.cc:107] Waiting for socket to connect
0809 09:32:57.459056 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 09:32:57.459163 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:32:57.459408 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:32:57.459411 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:32:57.459644 (+ 233us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:32:57.459790 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:32:57.459796 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:32:57.460813 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:32:57.460817 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:32:57.461777 (+ 960us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:32:57.461789 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:32:57.462398 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:32:57.462402 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:32:57.462524 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:57.462995 (+ 471us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 09:32:57.463011 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:32:57.463499 (+ 488us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 09:32:57.465557 (+ 2058us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:32:57.465561 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:32:57.465564 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:32:57.465774 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:32:57.466053 (+ 279us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:32:57.466057 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:32:57.466059 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:32:57.466143 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:32:57.466556 (+ 413us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 09:32:57.466560 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 09:32:57.466662 (+ 102us) client_negotiation.cc:769] Sending connection context
0809 09:32:57.466796 (+ 134us) client_negotiation.cc:240] Negotiation successful
0809 09:32:57.466968 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":490,"mutex_wait_us":45,"spinlock_wait_cycles":1408,"thread_start_us":104,"threads_started":1}
I20260809 09:32:57.467233 30997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:57.458736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49568 (local address 127.20.125.189:44285)
0809 09:32:57.458851 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0809 09:32:57.458858 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 09:32:57.459145 (+ 287us) server_negotiation.cc:407] Connection header received
0809 09:32:57.459259 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:32:57.459261 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:32:57.459302 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:32:57.459395 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:32:57.459907 (+ 512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:57.460688 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:57.461903 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:57.462271 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:57.462382 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:57.463650 (+ 1268us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:32:57.463653 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:32:57.463656 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:32:57.463681 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:32:57.465430 (+ 1749us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:32:57.465872 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:32:57.465878 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:32:57.465880 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:32:57.465962 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:32:57.466231 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:32:57.466237 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:32:57.466239 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:32:57.466421 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:32:57.466567 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0809 09:32:57.466841 (+ 274us) server_negotiation.cc:299] Negotiation successful
0809 09:32:57.467062 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":136,"server-negotiator.queue_time_us":52}
I20260809 09:32:57.467759 30241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:57.458864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52652 (local address 127.20.125.190:37175)
0809 09:32:57.459003 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0809 09:32:57.459006 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 09:32:57.459018 (+ 12us) server_negotiation.cc:407] Connection header received
0809 09:32:57.459052 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:32:57.459053 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:32:57.459090 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:32:57.459190 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:32:57.459957 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:57.460732 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:57.461592 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:57.461824 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:57.461938 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:57.464042 (+ 2104us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:32:57.464046 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:32:57.464048 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:32:57.464078 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:32:57.465772 (+ 1694us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:32:57.466296 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:32:57.466300 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:32:57.466302 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:32:57.466393 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:32:57.466742 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:32:57.466748 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:32:57.466753 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:32:57.466988 (+ 235us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:32:57.467086 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0809 09:32:57.467454 (+ 368us) server_negotiation.cc:299] Negotiation successful
0809 09:32:57.467591 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260809 09:32:57.468228 31030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:57.458349 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:37175 (local address 127.0.0.1:52652)
0809 09:32:57.458697 (+ 348us) negotiation.cc:107] Waiting for socket to connect
0809 09:32:57.458715 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 09:32:57.458829 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:32:57.459311 (+ 482us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:32:57.459314 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:32:57.459625 (+ 311us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:32:57.459798 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:32:57.459807 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:32:57.460860 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:32:57.460862 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:32:57.461465 (+ 603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:32:57.461473 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:32:57.462068 (+ 595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:32:57.462072 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:32:57.462234 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:57.463361 (+ 1127us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 09:32:57.463394 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:32:57.463886 (+ 492us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 09:32:57.465912 (+ 2026us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:32:57.465916 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:32:57.465917 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:32:57.466163 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:32:57.466543 (+ 380us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:32:57.466546 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:32:57.466548 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:32:57.466639 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:32:57.467132 (+ 493us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 09:32:57.467134 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 09:32:57.467233 (+ 99us) client_negotiation.cc:769] Sending connection context
0809 09:32:57.467909 (+ 676us) client_negotiation.cc:240] Negotiation successful
0809 09:32:57.468074 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":244,"mutex_wait_us":61,"thread_start_us":133,"threads_started":1}
I20260809 09:32:57.471516 31003 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3c029f42802647dea9a4c2f436be31eb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 09:32:57.471661 31003 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3c029f42802647dea9a4c2f436be31eb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec1960de545a42ba9ac3e3e9adc99056" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 37175 } } peers { permanent_uuid: "3c029f42802647dea9a4c2f436be31eb" member_type: VOTER last_known_addr { host: "127.20.125.189" port: 44285 } }
I20260809 09:32:57.472539 31003 leader_election.cc:288] T 00000000000000000000000000000000 P 3c029f42802647dea9a4c2f436be31eb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ec1960de545a42ba9ac3e3e9adc99056 (127.20.125.190:37175)
I20260809 09:32:57.473062 30194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3c029f42802647dea9a4c2f436be31eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec1960de545a42ba9ac3e3e9adc99056" is_pre_election: true
I20260809 09:32:57.474364 30922 leader_election.cc:302] T 00000000000000000000000000000000 P 3c029f42802647dea9a4c2f436be31eb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 2 responses out of 2 voters: 1 yes votes; 1 no votes. yes voters: 3c029f42802647dea9a4c2f436be31eb; no voters: ec1960de545a42ba9ac3e3e9adc99056
I20260809 09:32:57.474839 31003 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 3c029f42802647dea9a4c2f436be31eb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
Aug 09 09:32:57 dist-test-slave-7kzw krb5kdc[30113](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786267976, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.125.190@KRBTEST.COM for oryx/127.20.125.189@KRBTEST.COM
I20260809 09:32:57.483690 31032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:57.473553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.189:44285 (local address 127.0.0.1:49584)
0809 09:32:57.473906 (+ 353us) negotiation.cc:107] Waiting for socket to connect
0809 09:32:57.473946 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 09:32:57.474100 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:32:57.474801 (+ 701us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:32:57.474808 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:32:57.475335 (+ 527us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:32:57.475480 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:32:57.475488 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:32:57.476190 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:32:57.476193 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:32:57.477116 (+ 923us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:32:57.477135 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:32:57.477601 (+ 466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:32:57.477605 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:32:57.477727 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:57.478264 (+ 537us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 09:32:57.478277 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:32:57.480010 (+ 1733us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 09:32:57.481979 (+ 1969us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:32:57.481988 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:32:57.481994 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:32:57.482239 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:32:57.482581 (+ 342us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:32:57.482583 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:32:57.482585 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:32:57.482730 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:32:57.483155 (+ 425us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 09:32:57.483160 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 09:32:57.483275 (+ 115us) client_negotiation.cc:769] Sending connection context
0809 09:32:57.483397 (+ 122us) client_negotiation.cc:240] Negotiation successful
0809 09:32:57.483552 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":102,"threads_started":1}
I20260809 09:32:57.483728 30997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:57.473580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49584 (local address 127.20.125.189:44285)
0809 09:32:57.473705 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0809 09:32:57.473710 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 09:32:57.474024 (+ 314us) server_negotiation.cc:407] Connection header received
0809 09:32:57.474586 (+ 562us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:32:57.474590 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:32:57.474645 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:32:57.474736 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:32:57.475594 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:57.476069 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:57.477259 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:57.477489 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:57.477597 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:57.480165 (+ 2568us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:32:57.480169 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:32:57.480172 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:32:57.480197 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:32:57.481862 (+ 1665us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:32:57.482340 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:32:57.482344 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:32:57.482346 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:32:57.482434 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:32:57.482827 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:32:57.482830 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:32:57.482831 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:32:57.483043 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:32:57.483143 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0809 09:32:57.483476 (+ 333us) server_negotiation.cc:299] Negotiation successful
0809 09:32:57.483590 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260809 09:32:57.485337 30181 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.20.125.189:44285: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "3c029f42802647dea9a4c2f436be31eb"
member_type: NON_VOTER
last_known_addr {
host: "127.20.125.189"
port: 44285
}
attrs {
promote: true
}
}
dest_uuid: "ec1960de545a42ba9ac3e3e9adc99056"
cas_config_opid_index: -1
I20260809 09:32:57.486038 30181 consensus_queue.cc:237] T 00000000000000000000000000000000 P ec1960de545a42ba9ac3e3e9adc99056 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "ec1960de545a42ba9ac3e3e9adc99056" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 37175 } } peers { permanent_uuid: "3c029f42802647dea9a4c2f436be31eb" member_type: NON_VOTER last_known_addr { host: "127.20.125.189" port: 44285 } attrs { promote: true } }
I20260809 09:32:57.488168 31038 raft_consensus.cc:2932] T 00000000000000000000000000000000 P ec1960de545a42ba9ac3e3e9adc99056 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 3c029f42802647dea9a4c2f436be31eb (127.20.125.189) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "ec1960de545a42ba9ac3e3e9adc99056" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 37175 } } peers { permanent_uuid: "3c029f42802647dea9a4c2f436be31eb" member_type: NON_VOTER last_known_addr { host: "127.20.125.189" port: 44285 } attrs { promote: true } } }
I20260809 09:32:57.488945 30957 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3c029f42802647dea9a4c2f436be31eb [term 0 FOLLOWER]: Advancing to term 1
WARNING: no policy specified for oryx/127.20.125.188@KRBTEST.COM; defaulting to no policy
I20260809 09:32:57.489894 31038 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.20.125.189:44285
I20260809 09:32:57.490406 30901 master.cc:537] Master@127.20.125.189:44285 shutting down...
I20260809 09:32:57.490638 30957 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 3c029f42802647dea9a4c2f436be31eb [term 1 FOLLOWER]: Refusing update from remote peer ec1960de545a42ba9ac3e3e9adc99056: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260809 09:32:57.491447 31042 consensus_queue.cc:1035] T 00000000000000000000000000000000 P ec1960de545a42ba9ac3e3e9adc99056 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c029f42802647dea9a4c2f436be31eb" member_type: NON_VOTER last_known_addr { host: "127.20.125.189" port: 44285 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.20.125.188@KRBTEST.COM" created.
I20260809 09:32:57.493310 31041 sys_catalog.cc:455] T 00000000000000000000000000000000 P ec1960de545a42ba9ac3e3e9adc99056 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "ec1960de545a42ba9ac3e3e9adc99056" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "ec1960de545a42ba9ac3e3e9adc99056" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 37175 } } peers { permanent_uuid: "3c029f42802647dea9a4c2f436be31eb" member_type: NON_VOTER last_known_addr { host: "127.20.125.189" port: 44285 } attrs { promote: true } } }
I20260809 09:32:57.493429 31041 sys_catalog.cc:458] T 00000000000000000000000000000000 P ec1960de545a42ba9ac3e3e9adc99056 [sys.catalog]: This master's current role is: LEADER
W20260809 09:32:57.495332 30163 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
I20260809 09:32:57.496234 30901 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 3c029f42802647dea9a4c2f436be31eb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 09:32:57.496528 30901 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 3c029f42802647dea9a4c2f436be31eb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 09:32:57.496647 30901 tablet_replica.cc:330] T 00000000000000000000000000000000 P 3c029f42802647dea9a4c2f436be31eb: stopping tablet replica
W20260809 09:32:57.499888 30925 connection.cc:441] server connection from 127.0.0.1:49584 torn down before Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49584 (request call id 3) could send its response
W20260809 09:32:57.500171 30163 consensus_peers.cc:487] T 00000000000000000000000000000000 P ec1960de545a42ba9ac3e3e9adc99056 -> Peer 3c029f42802647dea9a4c2f436be31eb (127.20.125.189:44285): Couldn't send request to peer 3c029f42802647dea9a4c2f436be31eb. Status: Network error: recv got EOF from 127.20.125.189:44285 (error 108). This is attempt 1: this message will repeat every 5th retry.
I20260809 09:32:57.511804 30901 master.cc:552] Master@127.20.125.189:44285 shutdown complete.
I20260809 09:32:57.511880 30901 master_runner.cc:305] Clearing existing system tablet
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.20.125.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/krb5kdc/oryx_127.20.125.188.keytab.
Entry for principal oryx/127.20.125.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/krb5kdc/oryx_127.20.125.188.keytab.
I20260809 09:32:57.512169 30901 fs_manager.cc:359] Metadata directory not provided
I20260809 09:32:57.512245 30901 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 09:32:57.513252 20982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
/tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.125.188:36019
--webserver_interface=127.20.125.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/krb5kdc/oryx_127.20.125.188.keytab
--principal=oryx/127.20.125.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.125.148:36081
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.125.190:37175,127.20.125.189:44285,127.20.125.188:36019
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 09:32:57.513576 30901 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 09:32:57.514500 31054 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 09:32:57.514706 30901 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 09:32:57.514842 30901 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 09:32:57.514887 30901 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/minicluster-data/master-1/data,/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/minicluster-data/master-1/wal
uuid: "3c029f42802647dea9a4c2f436be31eb"
format_stamp: "Formatted at 2026-08-09 09:32:57 on dist-test-slave-7kzw"
server_key: "fb7011462a080d57d7b61308f2cdf82a"
server_key_iv: "75126c7a7cdbf0258f06edb43de58aa7"
server_key_version: "encryptionkey@0"
I20260809 09:32:57.516651 30901 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 3c029f42802647dea9a4c2f436be31eb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 09:32:57.519301 30901 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 3c029f42802647dea9a4c2f436be31eb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260809 09:32:57.519410 30901 log.cc:1196] T 00000000000000000000000000000000 P 3c029f42802647dea9a4c2f436be31eb: Deleting WAL directory at /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260809 09:32:57.519654 30901 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 3c029f42802647dea9a4c2f436be31eb: Deleting consensus metadata
I20260809 09:32:57.519850 30901 master_runner.cc:315] Copying system tablet from 127.20.125.190:37175
I20260809 09:32:57.521028 30901 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 3c029f42802647dea9a4c2f436be31eb: tablet copy: Beginning tablet copy session from remote peer at address 127.20.125.190:37175
I20260809 09:32:57.529083 30241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:57.521713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52666 (local address 127.20.125.190:37175)
0809 09:32:57.521854 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0809 09:32:57.521860 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 09:32:57.522049 (+ 189us) server_negotiation.cc:407] Connection header received
0809 09:32:57.522195 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:32:57.522198 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:32:57.522259 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:32:57.522348 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:32:57.522882 (+ 534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:57.523418 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:57.524256 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:57.524485 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:57.524595 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:57.525764 (+ 1169us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:32:57.525768 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:32:57.525771 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:32:57.525797 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:32:57.527351 (+ 1554us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:32:57.527807 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:32:57.527811 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:32:57.527813 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:32:57.527902 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:32:57.528185 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:32:57.528189 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:32:57.528191 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:32:57.528382 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:32:57.528492 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0809 09:32:57.528833 (+ 341us) server_negotiation.cc:299] Negotiation successful
0809 09:32:57.528960 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 09:32:57.529587 31063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:57.521620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:37175 (local address 127.0.0.1:52666)
0809 09:32:57.521952 (+ 332us) negotiation.cc:107] Waiting for socket to connect
0809 09:32:57.521964 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 09:32:57.522097 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:32:57.522365 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:32:57.522369 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:32:57.522641 (+ 272us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:32:57.522754 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:32:57.522760 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:32:57.523531 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:32:57.523537 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:32:57.524117 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:32:57.524125 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:32:57.524599 (+ 474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:32:57.524603 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:32:57.524698 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:57.525282 (+ 584us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 09:32:57.525294 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:32:57.525616 (+ 322us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 09:32:57.527484 (+ 1868us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:32:57.527487 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:32:57.527490 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:32:57.527695 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:32:57.527995 (+ 300us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:32:57.527999 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:32:57.528001 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:32:57.528091 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:32:57.528482 (+ 391us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 09:32:57.528485 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 09:32:57.528589 (+ 104us) client_negotiation.cc:769] Sending connection context
0809 09:32:57.529262 (+ 673us) client_negotiation.cc:240] Negotiation successful
0809 09:32:57.529409 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":261,"thread_start_us":106,"threads_started":1}
I20260809 09:32:57.530190 30210 tablet_copy_service.cc:140] P ec1960de545a42ba9ac3e3e9adc99056: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 3c029f42802647dea9a4c2f436be31eb ({username='oryx', principal='oryx/127.20.125.189@KRBTEST.COM'} at 127.0.0.1:52666)
I20260809 09:32:57.530346 30210 tablet_copy_service.cc:161] P ec1960de545a42ba9ac3e3e9adc99056: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 3c029f42802647dea9a4c2f436be31eb at {username='oryx', principal='oryx/127.20.125.189@KRBTEST.COM'} at 127.0.0.1:52666: session id = 3c029f42802647dea9a4c2f436be31eb-00000000000000000000000000000000
I20260809 09:32:57.531867 30210 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P ec1960de545a42ba9ac3e3e9adc99056: Tablet Copy: opened 0 blocks and 1 log segments
I20260809 09:32:57.533344 30901 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260809 09:32:57.535259 30901 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 3c029f42802647dea9a4c2f436be31eb: tablet copy: Starting download of 0 data blocks...
I20260809 09:32:57.535470 30901 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 3c029f42802647dea9a4c2f436be31eb: tablet copy: Starting download of 1 WAL segments...
I20260809 09:32:57.537164 30901 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 3c029f42802647dea9a4c2f436be31eb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260809 09:32:57.538781 30210 tablet_copy_service.cc:342] P ec1960de545a42ba9ac3e3e9adc99056: Request end of tablet copy session 3c029f42802647dea9a4c2f436be31eb-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.20.125.189@KRBTEST.COM'} at 127.0.0.1:52666
I20260809 09:32:57.538890 30210 tablet_copy_service.cc:433] P ec1960de545a42ba9ac3e3e9adc99056: ending tablet copy session 3c029f42802647dea9a4c2f436be31eb-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 3c029f42802647dea9a4c2f436be31eb
I20260809 09:32:57.546332 30901 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 09:32:57.548242 31064 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:32:57.548367 31068 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:32:57.548614 30901 server_base.cc:1034] running on GCE node
W20260809 09:32:57.548911 31066 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:32:57.549152 30901 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 09:32:57.549474 30901 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 09:32:57.550652 30901 hybrid_clock.cc:648] HybridClock initialized: now 1786267977550659 us; error 42 us; skew 500 ppm
Aug 09 09:32:57 dist-test-slave-7kzw krb5kdc[30113](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786267977, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.125.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 09:32:57.552270 30901 init.cc:377] Logged in from keytab as oryx/127.20.125.189@KRBTEST.COM (short username oryx)
I20260809 09:32:57.552769 30901 webserver.cc:460] Webserver started at http://127.20.125.189:42933/ using document root <none> and password file <none>
I20260809 09:32:57.552984 30901 fs_manager.cc:359] Metadata directory not provided
I20260809 09:32:57.553041 30901 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 09:32:57.554318 30901 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 09:32:57.555358 31085 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 09:32:57.555598 30901 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 09:32:57.555671 30901 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/minicluster-data/master-1/data,/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/minicluster-data/master-1/wal
uuid: "3c029f42802647dea9a4c2f436be31eb"
format_stamp: "Formatted at 2026-08-09 09:32:57 on dist-test-slave-7kzw"
server_key: "fb7011462a080d57d7b61308f2cdf82a"
server_key_iv: "75126c7a7cdbf0258f06edb43de58aa7"
server_key_version: "encryptionkey@0"
I20260809 09:32:57.555755 30901 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 09:32:57.566694 30901 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 09:32:57.568826 30901 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 09:32:57.574014 30901 rpc_server.cc:307] RPC server started. Bound to: 127.20.125.189:44285
I20260809 09:32:57.574239 31166 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.125.189:44285 every 8 connection(s)
I20260809 09:32:57.574733 30901 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/minicluster-data/master-1/data/info.pb
I20260809 09:32:57.576292 31168 sys_catalog.cc:263] Verifying existing consensus state
I20260809 09:32:57.577134 31168 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3c029f42802647dea9a4c2f436be31eb: Bootstrap starting.
I20260809 09:32:57.585355 31168 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3c029f42802647dea9a4c2f436be31eb: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 09:32:57.585676 31168 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3c029f42802647dea9a4c2f436be31eb: Bootstrap complete.
I20260809 09:32:57.586174 31168 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3c029f42802647dea9a4c2f436be31eb [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "ec1960de545a42ba9ac3e3e9adc99056" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 37175 } } peers { permanent_uuid: "3c029f42802647dea9a4c2f436be31eb" member_type: NON_VOTER last_known_addr { host: "127.20.125.189" port: 44285 } attrs { promote: true } }
I20260809 09:32:57.586314 31168 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3c029f42802647dea9a4c2f436be31eb [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 3c029f42802647dea9a4c2f436be31eb, State: Initialized, Role: LEARNER
I20260809 09:32:57.586431 31168 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3c029f42802647dea9a4c2f436be31eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "ec1960de545a42ba9ac3e3e9adc99056" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 37175 } } peers { permanent_uuid: "3c029f42802647dea9a4c2f436be31eb" member_type: NON_VOTER last_known_addr { host: "127.20.125.189" port: 44285 } attrs { promote: true } }
I20260809 09:32:57.586829 31171 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3c029f42802647dea9a4c2f436be31eb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "ec1960de545a42ba9ac3e3e9adc99056" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 37175 } } peers { permanent_uuid: "3c029f42802647dea9a4c2f436be31eb" member_type: NON_VOTER last_known_addr { host: "127.20.125.189" port: 44285 } attrs { promote: true } } }
I20260809 09:32:57.586994 31171 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3c029f42802647dea9a4c2f436be31eb [sys.catalog]: This master's current role is: LEARNER
I20260809 09:32:57.586993 31168 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3c029f42802647dea9a4c2f436be31eb [sys.catalog]: configured and running, proceeding with master startup.
I20260809 09:32:57.590765 31197 catalog_manager.cc:1259] Loaded cluster ID: a876011cffd24bdd84e8602f372dcb24
I20260809 09:32:57.590873 31197 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 3c029f42802647dea9a4c2f436be31eb: loading cluster ID for follower catalog manager: success
I20260809 09:32:57.592655 31197 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 3c029f42802647dea9a4c2f436be31eb: acquiring CA information for follower catalog manager: success
I20260809 09:32:57.593860 31197 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 3c029f42802647dea9a4c2f436be31eb: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260809 09:32:57.675210 31055 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 09:32:57.675472 31055 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 09:32:57.675513 31055 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 09:32:57.679630 31055 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 09:32:57.679733 31055 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 09:32:57.679756 31055 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 09:32:57.679800 31055 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 09:32:57.679832 31055 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 09:32:57.685137 31055 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.125.148:36081
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.20.125.190:37175,127.20.125.189:44285,127.20.125.188:36019
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/krb5kdc/oryx_127.20.125.188.keytab
--principal=oryx/127.20.125.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.125.188:36019
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.20.125.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31055
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 08:38:55 UTC on ccycloud.quasar-njjleu.root.comops.site
build id 4575935
I20260809 09:32:57.686206 31055 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 09:32:57.687295 31055 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 09:32:57.693970 30481 heartbeater.cc:503] Master 127.20.125.190:37175 was elected leader, sending a full tablet report...
W20260809 09:32:57.695931 31203 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:32:57.695936 31202 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:32:57.696126 31208 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:32:57.696467 31055 server_base.cc:1034] running on GCE node
I20260809 09:32:57.697132 31055 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 09:32:57.698532 31055 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 09:32:57.699748 31055 hybrid_clock.cc:648] HybridClock initialized: now 1786267977699732 us; error 32 us; skew 500 ppm
Aug 09 09:32:57 dist-test-slave-7kzw krb5kdc[30113](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786267977, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.125.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 09:32:57.702922 31055 init.cc:377] Logged in from keytab as oryx/127.20.125.188@KRBTEST.COM (short username oryx)
I20260809 09:32:57.703859 31055 webserver.cc:460] Webserver started at http://127.20.125.188:37097/ using document root <none> and password file <none>
I20260809 09:32:57.704448 31055 fs_manager.cc:359] Metadata directory not provided
I20260809 09:32:57.704545 31055 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 09:32:57.704790 31055 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 09:32:57.707101 31055 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/minicluster-data/master-2/data/instance:
uuid: "dece4cc043764f62a3667954979c5562"
format_stamp: "Formatted at 2026-08-09 09:32:57 on dist-test-slave-7kzw"
server_key: "a2e68086116e8cf055924a594ccc5357"
server_key_iv: "95a697b4f70c6b73bffc7a60e6c85757"
server_key_version: "encryptionkey@0"
I20260809 09:32:57.707721 31055 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/minicluster-data/master-2/wal/instance:
uuid: "dece4cc043764f62a3667954979c5562"
format_stamp: "Formatted at 2026-08-09 09:32:57 on dist-test-slave-7kzw"
server_key: "a2e68086116e8cf055924a594ccc5357"
server_key_iv: "95a697b4f70c6b73bffc7a60e6c85757"
server_key_version: "encryptionkey@0"
I20260809 09:32:57.711702 31055 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260809 09:32:57.714154 31217 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 09:32:57.715447 31055 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 09:32:57.715597 31055 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/minicluster-data/master-2/data,/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/minicluster-data/master-2/wal
uuid: "dece4cc043764f62a3667954979c5562"
format_stamp: "Formatted at 2026-08-09 09:32:57 on dist-test-slave-7kzw"
server_key: "a2e68086116e8cf055924a594ccc5357"
server_key_iv: "95a697b4f70c6b73bffc7a60e6c85757"
server_key_version: "encryptionkey@0"
I20260809 09:32:57.715714 31055 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 09:32:57.737910 31055 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 09:32:57.741649 31055 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 09:32:57.741882 31055 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 09:32:57.750304 31055 rpc_server.cc:307] RPC server started. Bound to: 127.20.125.188:36019
I20260809 09:32:57.750677 31299 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.125.188:36019 every 8 connection(s)
I20260809 09:32:57.751405 31055 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/minicluster-data/master-2/data/info.pb
I20260809 09:32:57.754685 31300 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 09:32:57.757936 20982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu as pid 31055
I20260809 09:32:57.759125 31300 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.125.190" port: 37175 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 09:32:57 dist-test-slave-7kzw krb5kdc[30113](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786267977, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.125.188@KRBTEST.COM for oryx/127.20.125.190@KRBTEST.COM
Aug 09 09:32:57 dist-test-slave-7kzw krb5kdc[30113](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786267976, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.20.125.188@KRBTEST.COM
I20260809 09:32:57.772444 30241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:57.761114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52680 (local address 127.20.125.190:37175)
0809 09:32:57.761275 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0809 09:32:57.761279 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 09:32:57.761468 (+ 189us) server_negotiation.cc:407] Connection header received
0809 09:32:57.761647 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:32:57.761651 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:32:57.761718 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:32:57.761767 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:32:57.762755 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:57.763545 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:57.764683 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:57.764915 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:57.765057 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:57.768036 (+ 2979us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:32:57.768040 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:32:57.768043 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:32:57.768075 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:32:57.769964 (+ 1889us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:32:57.770760 (+ 796us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:32:57.770764 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:32:57.770766 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:32:57.770870 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:32:57.771277 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:32:57.771280 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:32:57.771282 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:32:57.771493 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:32:57.771614 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0809 09:32:57.772088 (+ 474us) server_negotiation.cc:299] Negotiation successful
0809 09:32:57.772225 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260809 09:32:57.773350 31303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:57.760953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:37175 (local address 127.0.0.1:52680)
0809 09:32:57.761312 (+ 359us) negotiation.cc:107] Waiting for socket to connect
0809 09:32:57.761362 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0809 09:32:57.761500 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:32:57.761927 (+ 427us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:32:57.761932 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:32:57.762298 (+ 366us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:32:57.762609 (+ 311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:32:57.762622 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:32:57.763712 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:32:57.763714 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:32:57.764514 (+ 800us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:32:57.764523 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:32:57.765094 (+ 571us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:32:57.765098 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:32:57.765278 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:57.766159 (+ 881us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 09:32:57.766175 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:32:57.767850 (+ 1675us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 09:32:57.770089 (+ 2239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:32:57.770097 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:32:57.770106 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:32:57.770635 (+ 529us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:32:57.770983 (+ 348us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:32:57.770988 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:32:57.770990 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:32:57.771155 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:32:57.771611 (+ 456us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 09:32:57.771618 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 09:32:57.771839 (+ 221us) client_negotiation.cc:769] Sending connection context
0809 09:32:57.772022 (+ 183us) client_negotiation.cc:240] Negotiation successful
0809 09:32:57.772394 (+ 372us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":196,"mutex_wait_us":51,"spinlock_wait_cycles":8832,"thread_start_us":129,"threads_started":1}
I20260809 09:32:57.773351 31302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:57.759638 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37584 (local address 127.20.125.188:36019)
0809 09:32:57.760106 (+ 468us) server_negotiation.cc:206] Beginning negotiation
0809 09:32:57.760121 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 09:32:57.760158 (+ 37us) server_negotiation.cc:407] Connection header received
0809 09:32:57.760826 (+ 668us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:32:57.760843 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:32:57.761250 (+ 407us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:32:57.761620 (+ 370us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:32:57.762501 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:57.763770 (+ 1269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:57.764730 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:57.765155 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:57.765339 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:57.768553 (+ 3214us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:32:57.768559 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:32:57.768570 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:32:57.768622 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:32:57.770576 (+ 1954us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:32:57.771029 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:32:57.771034 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:32:57.771036 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:32:57.771146 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:32:57.771438 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:32:57.771442 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:32:57.771444 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:32:57.771824 (+ 380us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:32:57.772034 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0809 09:32:57.772278 (+ 244us) server_negotiation.cc:299] Negotiation successful
0809 09:32:57.772459 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":41,"server-negotiator.queue_time_us":327,"thread_start_us":157,"threads_started":1}
I20260809 09:32:57.774994 31300 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.125.189" port: 44285 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 09:32:57 dist-test-slave-7kzw krb5kdc[30113](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786267977, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.125.188@KRBTEST.COM for oryx/127.20.125.189@KRBTEST.COM
I20260809 09:32:57.784291 31303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:57.775511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.189:44285 (local address 127.0.0.1:49600)
0809 09:32:57.775959 (+ 448us) negotiation.cc:107] Waiting for socket to connect
0809 09:32:57.775972 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 09:32:57.776040 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:32:57.776320 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:32:57.776323 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:32:57.776459 (+ 136us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:32:57.776548 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:32:57.776554 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:32:57.777672 (+ 1118us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:32:57.777674 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:32:57.778199 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:32:57.778207 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:32:57.778827 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:32:57.778830 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:32:57.778911 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:57.779343 (+ 432us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 09:32:57.779356 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:32:57.780730 (+ 1374us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 09:32:57.782842 (+ 2112us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:32:57.782845 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:32:57.782846 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:32:57.783058 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:32:57.783358 (+ 300us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:32:57.783361 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:32:57.783362 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:32:57.783448 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:32:57.783875 (+ 427us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 09:32:57.783877 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 09:32:57.783960 (+ 83us) client_negotiation.cc:769] Sending connection context
0809 09:32:57.784054 (+ 94us) client_negotiation.cc:240] Negotiation successful
0809 09:32:57.784170 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":379}
I20260809 09:32:57.784386 31304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:57.775682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49600 (local address 127.20.125.189:44285)
0809 09:32:57.775964 (+ 282us) server_negotiation.cc:206] Beginning negotiation
0809 09:32:57.775968 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 09:32:57.776007 (+ 39us) server_negotiation.cc:407] Connection header received
0809 09:32:57.776138 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:32:57.776141 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:32:57.776186 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:32:57.776266 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:32:57.776778 (+ 512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:57.777511 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:57.778331 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:57.778701 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:57.778799 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:57.780896 (+ 2097us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:32:57.780899 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:32:57.780901 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:32:57.780926 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:32:57.782732 (+ 1806us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:32:57.783159 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:32:57.783162 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:32:57.783163 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:32:57.783248 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:32:57.783538 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:32:57.783540 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:32:57.783542 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:32:57.783756 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:32:57.783841 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0809 09:32:57.784147 (+ 306us) server_negotiation.cc:299] Negotiation successful
0809 09:32:57.784264 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":204,"thread_start_us":94,"threads_started":1}
I20260809 09:32:57.785378 31300 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.125.188" port: 36019 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 09:32:57 dist-test-slave-7kzw krb5kdc[30113](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786267977, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.125.188@KRBTEST.COM for oryx/127.20.125.188@KRBTEST.COM
I20260809 09:32:57.793295 31302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:57.785886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37592 (local address 127.20.125.188:36019)
0809 09:32:57.786004 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0809 09:32:57.786007 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 09:32:57.786019 (+ 12us) server_negotiation.cc:407] Connection header received
0809 09:32:57.786076 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:32:57.786078 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:32:57.786109 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:32:57.786177 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:32:57.786593 (+ 416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:57.787074 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:57.787823 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:57.788046 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:57.788167 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:57.790298 (+ 2131us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:32:57.790302 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:32:57.790304 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:32:57.790331 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:32:57.791673 (+ 1342us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:32:57.792077 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:32:57.792082 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:32:57.792084 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:32:57.792161 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:32:57.792446 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:32:57.792449 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:32:57.792450 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:32:57.792620 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:32:57.792749 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0809 09:32:57.792991 (+ 242us) server_negotiation.cc:299] Negotiation successful
0809 09:32:57.793136 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":45,"server-negotiator.queue_time_us":51}
I20260809 09:32:57.793295 31303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:57.785810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.188:36019 (local address 127.0.0.1:37592)
0809 09:32:57.785934 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0809 09:32:57.785950 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 09:32:57.786021 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:32:57.786205 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:32:57.786207 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:32:57.786336 (+ 129us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:32:57.786401 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:32:57.786406 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:32:57.787189 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:32:57.787196 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:32:57.787695 (+ 499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:32:57.787703 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:32:57.788166 (+ 463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:32:57.788170 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:32:57.788241 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:57.788730 (+ 489us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 09:32:57.788742 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:32:57.790161 (+ 1419us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 09:32:57.791785 (+ 1624us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:32:57.791791 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:32:57.791793 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:32:57.791979 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:32:57.792262 (+ 283us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:32:57.792266 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:32:57.792268 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:32:57.792349 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:32:57.792738 (+ 389us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 09:32:57.792741 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 09:32:57.792831 (+ 90us) client_negotiation.cc:769] Sending connection context
0809 09:32:57.792936 (+ 105us) client_negotiation.cc:240] Negotiation successful
0809 09:32:57.793081 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":62}
I20260809 09:32:57.797015 31300 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dece4cc043764f62a3667954979c5562: Bootstrap starting.
I20260809 09:32:57.799746 31300 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dece4cc043764f62a3667954979c5562: Neither blocks nor log segments found. Creating new log.
I20260809 09:32:57.800603 31300 log.cc:824] T 00000000000000000000000000000000 P dece4cc043764f62a3667954979c5562: Log is configured to *not* fsync() on all Append() calls
I20260809 09:32:57.802627 31300 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dece4cc043764f62a3667954979c5562: No bootstrap required, opened a new log
I20260809 09:32:57.805783 31300 raft_consensus.cc:348] T 00000000000000000000000000000000 P dece4cc043764f62a3667954979c5562 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec1960de545a42ba9ac3e3e9adc99056" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 37175 } } peers { permanent_uuid: "3c029f42802647dea9a4c2f436be31eb" member_type: VOTER last_known_addr { host: "127.20.125.189" port: 44285 } } peers { permanent_uuid: "dece4cc043764f62a3667954979c5562" member_type: VOTER last_known_addr { host: "127.20.125.188" port: 36019 } }
I20260809 09:32:57.806016 31300 raft_consensus.cc:374] T 00000000000000000000000000000000 P dece4cc043764f62a3667954979c5562 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 09:32:57.806098 31300 raft_consensus.cc:729] T 00000000000000000000000000000000 P dece4cc043764f62a3667954979c5562 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dece4cc043764f62a3667954979c5562, State: Initialized, Role: FOLLOWER
I20260809 09:32:57.806627 31300 consensus_queue.cc:260] T 00000000000000000000000000000000 P dece4cc043764f62a3667954979c5562 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec1960de545a42ba9ac3e3e9adc99056" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 37175 } } peers { permanent_uuid: "3c029f42802647dea9a4c2f436be31eb" member_type: VOTER last_known_addr { host: "127.20.125.189" port: 44285 } } peers { permanent_uuid: "dece4cc043764f62a3667954979c5562" member_type: VOTER last_known_addr { host: "127.20.125.188" port: 36019 } }
I20260809 09:32:57.807101 31307 sys_catalog.cc:455] T 00000000000000000000000000000000 P dece4cc043764f62a3667954979c5562 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec1960de545a42ba9ac3e3e9adc99056" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 37175 } } peers { permanent_uuid: "3c029f42802647dea9a4c2f436be31eb" member_type: VOTER last_known_addr { host: "127.20.125.189" port: 44285 } } peers { permanent_uuid: "dece4cc043764f62a3667954979c5562" member_type: VOTER last_known_addr { host: "127.20.125.188" port: 36019 } } }
I20260809 09:32:57.807240 31307 sys_catalog.cc:458] T 00000000000000000000000000000000 P dece4cc043764f62a3667954979c5562 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 09:32:57.807855 31300 sys_catalog.cc:564] T 00000000000000000000000000000000 P dece4cc043764f62a3667954979c5562 [sys.catalog]: configured and running, proceeding with master startup.
W20260809 09:32:57.813016 31325 catalog_manager.cc:1558] T 00000000000000000000000000000000 P dece4cc043764f62a3667954979c5562: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 09:32:57.813112 31325 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 09:32:57.816994 31055 master_runner.cc:428] Detected that this master dece4cc043764f62a3667954979c5562 is joining an existing cluster
I20260809 09:32:57.817052 31055 master_runner.cc:432] Initiating AddMaster RPC to add 127.20.125.188:36019
I20260809 09:32:57.827663 20982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu with pid 30275
I20260809 09:32:57.830120 30241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:57.819336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52686 (local address 127.20.125.190:37175)
0809 09:32:57.819482 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0809 09:32:57.819487 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 09:32:57.820332 (+ 845us) server_negotiation.cc:407] Connection header received
0809 09:32:57.820447 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:32:57.820450 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:32:57.820508 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:32:57.820594 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:32:57.821288 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:57.822127 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:57.824553 (+ 2426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:57.824902 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:57.824979 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:57.826281 (+ 1302us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:32:57.826284 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:32:57.826287 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:32:57.826316 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:32:57.828219 (+ 1903us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:32:57.828742 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:32:57.828746 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:32:57.828748 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:32:57.828839 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:32:57.829136 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:32:57.829139 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:32:57.829141 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:32:57.829359 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:32:57.829433 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0809 09:32:57.829824 (+ 391us) server_negotiation.cc:299] Negotiation successful
0809 09:32:57.829950 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260809 09:32:57.830732 31336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:57.819277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:37175 (local address 127.0.0.1:52686)
0809 09:32:57.820212 (+ 935us) negotiation.cc:107] Waiting for socket to connect
0809 09:32:57.820227 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 09:32:57.820351 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:32:57.820741 (+ 390us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:32:57.820744 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:32:57.820989 (+ 245us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:32:57.821133 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:32:57.821142 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:32:57.822280 (+ 1138us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:32:57.822285 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:32:57.824230 (+ 1945us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:32:57.824236 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:32:57.825129 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:32:57.825132 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:32:57.825224 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:57.825648 (+ 424us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 09:32:57.825664 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:32:57.826111 (+ 447us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 09:32:57.828356 (+ 2245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:32:57.828360 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:32:57.828365 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:32:57.828614 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:32:57.828948 (+ 334us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:32:57.828952 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:32:57.828954 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:32:57.829039 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:32:57.829506 (+ 467us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 09:32:57.829509 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 09:32:57.829611 (+ 102us) client_negotiation.cc:769] Sending connection context
0809 09:32:57.830332 (+ 721us) client_negotiation.cc:240] Negotiation successful
0809 09:32:57.830506 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":846,"mutex_wait_us":24,"spinlock_wait_cycles":11648,"thread_start_us":202,"threads_started":1}
Aug 09 09:32:57 dist-test-slave-7kzw krb5kdc[30113](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786267976, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.125.190@KRBTEST.COM for oryx/127.20.125.188@KRBTEST.COM
I20260809 09:32:57.854426 20982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
/tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.125.129:42545
--local_ip_for_outbound_sockets=127.20.125.129
--tserver_master_addrs=127.20.125.190:37175
--webserver_port=35525
--webserver_interface=127.20.125.129
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/krb5kdc/oryx_127.20.125.129.keytab
--principal=oryx/127.20.125.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.125.148:36081
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.20.125.190:37175,127.20.125.189:44285
--tserver_master_addrs=127.20.125.190:37175,127.20.125.189:44285,127.20.125.188:36019 with env {KRB5CCNAME=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 09:32:57.856032 31335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:57.819835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.188:36019 (local address 127.0.0.1:37608)
0809 09:32:57.820882 (+ 1047us) negotiation.cc:107] Waiting for socket to connect
0809 09:32:57.820896 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 09:32:57.820995 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:32:57.821313 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:32:57.821316 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:32:57.821509 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:32:57.821608 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:32:57.821614 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:32:57.822570 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:32:57.822572 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:32:57.823589 (+ 1017us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:32:57.823599 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:32:57.824334 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:32:57.824337 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:32:57.824418 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:57.847750 (+ 23332us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 09:32:57.847768 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:32:57.851810 (+ 4042us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 09:32:57.854231 (+ 2421us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:32:57.854234 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:32:57.854236 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:32:57.854502 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:32:57.854817 (+ 315us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:32:57.854822 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:32:57.854826 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:32:57.854916 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:32:57.855435 (+ 519us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 09:32:57.855441 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 09:32:57.855542 (+ 101us) client_negotiation.cc:769] Sending connection context
0809 09:32:57.855664 (+ 122us) client_negotiation.cc:240] Negotiation successful
0809 09:32:57.855839 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":974,"mutex_wait_us":107,"thread_start_us":101,"threads_started":1}
I20260809 09:32:57.856309 31302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:57.819891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37608 (local address 127.20.125.188:36019)
0809 09:32:57.820090 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0809 09:32:57.820094 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 09:32:57.820949 (+ 855us) server_negotiation.cc:407] Connection header received
0809 09:32:57.821082 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:32:57.821087 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:32:57.821125 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:32:57.821176 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:32:57.821710 (+ 534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:57.822423 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:57.823713 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:57.824082 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:57.824190 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:57.851986 (+ 27796us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:32:57.851993 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:32:57.851998 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:32:57.852038 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:32:57.854108 (+ 2070us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:32:57.854612 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:32:57.854618 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:32:57.854622 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:32:57.854707 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:32:57.855007 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:32:57.855012 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:32:57.855017 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:32:57.855256 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:32:57.855369 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0809 09:32:57.855986 (+ 617us) server_negotiation.cc:299] Negotiation successful
0809 09:32:57.856115 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":319,"server-negotiator.queue_time_us":133}
I20260809 09:32:57.856684 31304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:57.819388 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49608 (local address 127.20.125.189:44285)
0809 09:32:57.819569 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0809 09:32:57.819574 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 09:32:57.820639 (+ 1065us) server_negotiation.cc:407] Connection header received
0809 09:32:57.820769 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:32:57.820774 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:32:57.820828 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:32:57.820907 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:32:57.821536 (+ 629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:57.822366 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:57.823728 (+ 1362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:57.824058 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:57.824178 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:57.825637 (+ 1459us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:32:57.825640 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:32:57.825642 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:32:57.825666 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:32:57.827490 (+ 1824us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:32:57.854295 (+ 26805us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:32:57.854299 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:32:57.854301 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:32:57.854415 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:32:57.854791 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:32:57.854794 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:32:57.854796 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:32:57.855146 (+ 350us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:32:57.855242 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0809 09:32:57.856364 (+ 1122us) server_negotiation.cc:299] Negotiation successful
0809 09:32:57.856517 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260809 09:32:57.855867 31334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:57.819329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.189:44285 (local address 127.0.0.1:49608)
0809 09:32:57.820545 (+ 1216us) negotiation.cc:107] Waiting for socket to connect
0809 09:32:57.820555 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 09:32:57.820662 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:32:57.821031 (+ 369us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:32:57.821035 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:32:57.821288 (+ 253us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:32:57.821399 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:32:57.821409 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:32:57.822522 (+ 1113us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:32:57.822528 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:32:57.823590 (+ 1062us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:32:57.823599 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:32:57.824190 (+ 591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:32:57.824193 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:32:57.824310 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:57.824894 (+ 584us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 09:32:57.824906 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:32:57.825279 (+ 373us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 09:32:57.847985 (+ 22706us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:32:57.847988 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:32:57.847990 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:32:57.854151 (+ 6161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:32:57.854575 (+ 424us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:32:57.854578 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:32:57.854580 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:32:57.854684 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:32:57.855287 (+ 603us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 09:32:57.855290 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 09:32:57.855386 (+ 96us) client_negotiation.cc:769] Sending connection context
0809 09:32:57.855488 (+ 102us) client_negotiation.cc:240] Negotiation successful
0809 09:32:57.855658 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1167,"mutex_wait_us":97,"thread_start_us":103,"threads_started":1}
I20260809 09:32:57.860205 31032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:57.835499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.188:36019 (local address 127.0.0.1:37624)
0809 09:32:57.835685 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0809 09:32:57.835699 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 09:32:57.835776 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:32:57.836832 (+ 1056us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:32:57.836839 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:32:57.837124 (+ 285us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:32:57.837309 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:32:57.837325 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:32:57.838417 (+ 1092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:32:57.838424 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:32:57.839706 (+ 1282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:32:57.839723 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:32:57.840364 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:32:57.840368 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:32:57.840470 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:57.841092 (+ 622us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 09:32:57.841108 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:32:57.843261 (+ 2153us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 09:32:57.858334 (+ 15073us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:32:57.858338 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:32:57.858341 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:32:57.858669 (+ 328us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:32:57.859022 (+ 353us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:32:57.859025 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:32:57.859027 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:32:57.859130 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:32:57.859659 (+ 529us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 09:32:57.859662 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 09:32:57.859775 (+ 113us) client_negotiation.cc:769] Sending connection context
0809 09:32:57.859882 (+ 107us) client_negotiation.cc:240] Negotiation successful
0809 09:32:57.860040 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106}
I20260809 09:32:57.860724 31337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:57.835629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37624 (local address 127.20.125.188:36019)
0809 09:32:57.836510 (+ 881us) server_negotiation.cc:206] Beginning negotiation
0809 09:32:57.836519 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 09:32:57.836533 (+ 14us) server_negotiation.cc:407] Connection header received
0809 09:32:57.836604 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:32:57.836607 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:32:57.836676 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:32:57.836769 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:32:57.837463 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:57.838282 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:57.839854 (+ 1572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:57.840217 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:57.840714 (+ 497us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:57.843439 (+ 2725us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:32:57.843445 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:32:57.843451 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:32:57.843481 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:32:57.858139 (+ 14658us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:32:57.858792 (+ 653us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:32:57.858797 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:32:57.858801 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:32:57.858908 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:32:57.859281 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:32:57.859288 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:32:57.859292 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:32:57.859539 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:32:57.859652 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0809 09:32:57.860010 (+ 358us) server_negotiation.cc:299] Negotiation successful
0809 09:32:57.860149 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":807,"thread_start_us":699,"threads_started":1}
I20260809 09:32:57.861896 30181 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.20.125.188:36019: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "dece4cc043764f62a3667954979c5562"
member_type: NON_VOTER
last_known_addr {
host: "127.20.125.188"
port: 36019
}
attrs {
promote: true
}
}
dest_uuid: "ec1960de545a42ba9ac3e3e9adc99056"
cas_config_opid_index: 5
I20260809 09:32:57.862725 30181 consensus_queue.cc:237] T 00000000000000000000000000000000 P ec1960de545a42ba9ac3e3e9adc99056 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "ec1960de545a42ba9ac3e3e9adc99056" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 37175 } } peers { permanent_uuid: "3c029f42802647dea9a4c2f436be31eb" member_type: NON_VOTER last_known_addr { host: "127.20.125.189" port: 44285 } attrs { promote: true } } peers { permanent_uuid: "dece4cc043764f62a3667954979c5562" member_type: NON_VOTER last_known_addr { host: "127.20.125.188" port: 36019 } attrs { promote: true } }
I20260809 09:32:57.864022 31042 raft_consensus.cc:2932] T 00000000000000000000000000000000 P ec1960de545a42ba9ac3e3e9adc99056 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER dece4cc043764f62a3667954979c5562 (127.20.125.188) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "ec1960de545a42ba9ac3e3e9adc99056" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 37175 } } peers { permanent_uuid: "3c029f42802647dea9a4c2f436be31eb" member_type: NON_VOTER last_known_addr { host: "127.20.125.189" port: 44285 } attrs { promote: true } } peers { permanent_uuid: "dece4cc043764f62a3667954979c5562" member_type: NON_VOTER last_known_addr { host: "127.20.125.188" port: 36019 } attrs { promote: true } } }
I20260809 09:32:57.865300 31256 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dece4cc043764f62a3667954979c5562 [term 0 FOLLOWER]: Advancing to term 1
I20260809 09:32:57.865630 31042 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.20.125.188:36019
I20260809 09:32:57.865823 31042 sys_catalog.cc:455] T 00000000000000000000000000000000 P ec1960de545a42ba9ac3e3e9adc99056 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "ec1960de545a42ba9ac3e3e9adc99056" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "ec1960de545a42ba9ac3e3e9adc99056" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 37175 } } peers { permanent_uuid: "3c029f42802647dea9a4c2f436be31eb" member_type: NON_VOTER last_known_addr { host: "127.20.125.189" port: 44285 } attrs { promote: true } } peers { permanent_uuid: "dece4cc043764f62a3667954979c5562" member_type: NON_VOTER last_known_addr { host: "127.20.125.188" port: 36019 } attrs { promote: true } } }
I20260809 09:32:57.865917 31042 sys_catalog.cc:458] T 00000000000000000000000000000000 P ec1960de545a42ba9ac3e3e9adc99056 [sys.catalog]: This master's current role is: LEADER
I20260809 09:32:57.866408 31055 master.cc:537] Master@127.20.125.188:36019 shutting down...
I20260809 09:32:57.867080 31256 raft_consensus.cc:1261] T 00000000000000000000000000000000 P dece4cc043764f62a3667954979c5562 [term 1 FOLLOWER]: Refusing update from remote peer ec1960de545a42ba9ac3e3e9adc99056: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260809 09:32:57.868172 31041 consensus_queue.cc:1035] T 00000000000000000000000000000000 P ec1960de545a42ba9ac3e3e9adc99056 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dece4cc043764f62a3667954979c5562" member_type: NON_VOTER last_known_addr { host: "127.20.125.188" port: 36019 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
W20260809 09:32:57.870002 30162 consensus_peers.cc:487] T 00000000000000000000000000000000 P ec1960de545a42ba9ac3e3e9adc99056 -> Peer dece4cc043764f62a3667954979c5562 (127.20.125.188:36019): Couldn't send request to peer dece4cc043764f62a3667954979c5562. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260809 09:32:57.872081 31055 raft_consensus.cc:2229] T 00000000000000000000000000000000 P dece4cc043764f62a3667954979c5562 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 09:32:57.872471 31055 raft_consensus.cc:2258] T 00000000000000000000000000000000 P dece4cc043764f62a3667954979c5562 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 09:32:57.872956 31055 tablet_replica.cc:330] T 00000000000000000000000000000000 P dece4cc043764f62a3667954979c5562: stopping tablet replica
I20260809 09:32:57.875195 31032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:57.864356 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.189:44285 (local address 127.0.0.1:49622)
0809 09:32:57.869190 (+ 4834us) negotiation.cc:107] Waiting for socket to connect
0809 09:32:57.869208 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 09:32:57.869327 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:32:57.869855 (+ 528us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:32:57.869861 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:32:57.869886 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 09:32:57.870016 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:32:57.870026 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:32:57.871734 (+ 1708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:32:57.871738 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:32:57.873204 (+ 1466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:32:57.873230 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:32:57.874419 (+ 1189us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:32:57.874422 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:32:57.874668 (+ 246us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:57.874725 (+ 57us) client_negotiation.cc:769] Sending connection context
0809 09:32:57.874807 (+ 82us) client_negotiation.cc:240] Negotiation successful
0809 09:32:57.874886 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4736}
I20260809 09:32:57.875694 31304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:57.864424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49622 (local address 127.20.125.189:44285)
0809 09:32:57.867776 (+ 3352us) server_negotiation.cc:206] Beginning negotiation
0809 09:32:57.867781 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 09:32:57.869434 (+ 1653us) server_negotiation.cc:407] Connection header received
0809 09:32:57.869498 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:32:57.869504 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:32:57.869586 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:32:57.869689 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 09:32:57.870690 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:57.871592 (+ 902us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:57.873392 (+ 1800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:57.874071 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:57.874190 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:57.874350 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0809 09:32:57.875492 (+ 1142us) server_negotiation.cc:299] Negotiation successful
0809 09:32:57.875574 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3240}
I20260809 09:32:57.876196 31117 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 3c029f42802647dea9a4c2f436be31eb [term 1 LEARNER]: Refusing update from remote peer ec1960de545a42ba9ac3e3e9adc99056: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260809 09:32:57.876735 31041 consensus_queue.cc:1035] T 00000000000000000000000000000000 P ec1960de545a42ba9ac3e3e9adc99056 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c029f42802647dea9a4c2f436be31eb" member_type: NON_VOTER last_known_addr { host: "127.20.125.189" port: 44285 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260809 09:32:57.878682 31117 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 3c029f42802647dea9a4c2f436be31eb [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER dece4cc043764f62a3667954979c5562 (127.20.125.188) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "ec1960de545a42ba9ac3e3e9adc99056" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 37175 } } peers { permanent_uuid: "3c029f42802647dea9a4c2f436be31eb" member_type: NON_VOTER last_known_addr { host: "127.20.125.189" port: 44285 } attrs { promote: true } } peers { permanent_uuid: "dece4cc043764f62a3667954979c5562" member_type: NON_VOTER last_known_addr { host: "127.20.125.188" port: 36019 } attrs { promote: true } } }
I20260809 09:32:57.880043 31055 master.cc:552] Master@127.20.125.188:36019 shutdown complete.
I20260809 09:32:57.880095 31055 master_runner.cc:305] Clearing existing system tablet
I20260809 09:32:57.880388 31055 fs_manager.cc:359] Metadata directory not provided
I20260809 09:32:57.880461 31055 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 09:32:57.880982 31171 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3c029f42802647dea9a4c2f436be31eb [sys.catalog]: SysCatalogTable state changed. Reason: New leader ec1960de545a42ba9ac3e3e9adc99056. Latest consensus state: current_term: 1 leader_uuid: "ec1960de545a42ba9ac3e3e9adc99056" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "ec1960de545a42ba9ac3e3e9adc99056" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 37175 } } peers { permanent_uuid: "3c029f42802647dea9a4c2f436be31eb" member_type: NON_VOTER last_known_addr { host: "127.20.125.189" port: 44285 } attrs { promote: true } } peers { permanent_uuid: "dece4cc043764f62a3667954979c5562" member_type: NON_VOTER last_known_addr { host: "127.20.125.188" port: 36019 } attrs { promote: true } } }
I20260809 09:32:57.881111 31171 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3c029f42802647dea9a4c2f436be31eb [sys.catalog]: This master's current role is: LEARNER
I20260809 09:32:57.881210 31171 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3c029f42802647dea9a4c2f436be31eb [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ec1960de545a42ba9ac3e3e9adc99056" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "ec1960de545a42ba9ac3e3e9adc99056" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 37175 } } peers { permanent_uuid: "3c029f42802647dea9a4c2f436be31eb" member_type: NON_VOTER last_known_addr { host: "127.20.125.189" port: 44285 } attrs { promote: true } } peers { permanent_uuid: "dece4cc043764f62a3667954979c5562" member_type: NON_VOTER last_known_addr { host: "127.20.125.188" port: 36019 } attrs { promote: true } } }
I20260809 09:32:57.881285 31171 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3c029f42802647dea9a4c2f436be31eb [sys.catalog]: This master's current role is: LEARNER
I20260809 09:32:57.881747 31055 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 09:32:57.882201 31041 raft_consensus.cc:1050] T 00000000000000000000000000000000 P ec1960de545a42ba9ac3e3e9adc99056: attempting to promote NON_VOTER 3c029f42802647dea9a4c2f436be31eb to VOTER
I20260809 09:32:57.882730 31041 consensus_queue.cc:237] T 00000000000000000000000000000000 P ec1960de545a42ba9ac3e3e9adc99056 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "ec1960de545a42ba9ac3e3e9adc99056" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 37175 } } peers { permanent_uuid: "3c029f42802647dea9a4c2f436be31eb" member_type: VOTER last_known_addr { host: "127.20.125.189" port: 44285 } attrs { promote: false } } peers { permanent_uuid: "dece4cc043764f62a3667954979c5562" member_type: NON_VOTER last_known_addr { host: "127.20.125.188" port: 36019 } attrs { promote: true } }
I20260809 09:32:57.882917 31342 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 09:32:57.883113 31055 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 09:32:57.883235 31055 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 09:32:57.883277 31055 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/minicluster-data/master-2/data,/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/minicluster-data/master-2/wal
uuid: "dece4cc043764f62a3667954979c5562"
format_stamp: "Formatted at 2026-08-09 09:32:57 on dist-test-slave-7kzw"
server_key: "a2e68086116e8cf055924a594ccc5357"
server_key_iv: "95a697b4f70c6b73bffc7a60e6c85757"
server_key_version: "encryptionkey@0"
I20260809 09:32:57.883980 31032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:57.883620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.188:36019 (local address 127.0.0.1:37628)
0809 09:32:57.883835 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0809 09:32:57.883905 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.188:36019: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":146}
I20260809 09:32:57.884699 31032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:57.884450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.188:36019 (local address 127.0.0.1:37640)
0809 09:32:57.884573 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0809 09:32:57.884629 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.188:36019: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
W20260809 09:32:57.885066 30162 consensus_peers.cc:487] T 00000000000000000000000000000000 P ec1960de545a42ba9ac3e3e9adc99056 -> Peer dece4cc043764f62a3667954979c5562 (127.20.125.188:36019): Couldn't send request to peer dece4cc043764f62a3667954979c5562. Status: Network error: Client connection negotiation failed: client connection to 127.20.125.188:36019: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260809 09:32:57.885435 31055 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P dece4cc043764f62a3667954979c5562: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 09:32:57.885532 31117 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 3c029f42802647dea9a4c2f436be31eb [term 1 LEARNER]: Refusing update from remote peer ec1960de545a42ba9ac3e3e9adc99056: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260809 09:32:57.885921 31038 consensus_queue.cc:1035] T 00000000000000000000000000000000 P ec1960de545a42ba9ac3e3e9adc99056 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c029f42802647dea9a4c2f436be31eb" member_type: VOTER last_known_addr { host: "127.20.125.189" port: 44285 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260809 09:32:57.887713 31038 raft_consensus.cc:2932] T 00000000000000000000000000000000 P ec1960de545a42ba9ac3e3e9adc99056 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 3c029f42802647dea9a4c2f436be31eb (127.20.125.189) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "ec1960de545a42ba9ac3e3e9adc99056" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 37175 } } peers { permanent_uuid: "3c029f42802647dea9a4c2f436be31eb" member_type: VOTER last_known_addr { host: "127.20.125.189" port: 44285 } attrs { promote: false } } peers { permanent_uuid: "dece4cc043764f62a3667954979c5562" member_type: NON_VOTER last_known_addr { host: "127.20.125.188" port: 36019 } attrs { promote: true } } }
I20260809 09:32:57.888055 31055 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P dece4cc043764f62a3667954979c5562: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260809 09:32:57.888163 31055 log.cc:1196] T 00000000000000000000000000000000 P dece4cc043764f62a3667954979c5562: Deleting WAL directory at /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260809 09:32:57.888458 31055 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P dece4cc043764f62a3667954979c5562: Deleting consensus metadata
I20260809 09:32:57.888648 31055 master_runner.cc:315] Copying system tablet from 127.20.125.190:37175
I20260809 09:32:57.889130 31038 sys_catalog.cc:455] T 00000000000000000000000000000000 P ec1960de545a42ba9ac3e3e9adc99056 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "ec1960de545a42ba9ac3e3e9adc99056" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "ec1960de545a42ba9ac3e3e9adc99056" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 37175 } } peers { permanent_uuid: "3c029f42802647dea9a4c2f436be31eb" member_type: VOTER last_known_addr { host: "127.20.125.189" port: 44285 } attrs { promote: false } } peers { permanent_uuid: "dece4cc043764f62a3667954979c5562" member_type: NON_VOTER last_known_addr { host: "127.20.125.188" port: 36019 } attrs { promote: true } } }
I20260809 09:32:57.889246 31038 sys_catalog.cc:458] T 00000000000000000000000000000000 P ec1960de545a42ba9ac3e3e9adc99056 [sys.catalog]: This master's current role is: LEADER
I20260809 09:32:57.889369 31038 sys_catalog.cc:455] T 00000000000000000000000000000000 P ec1960de545a42ba9ac3e3e9adc99056 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ec1960de545a42ba9ac3e3e9adc99056" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "ec1960de545a42ba9ac3e3e9adc99056" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 37175 } } peers { permanent_uuid: "3c029f42802647dea9a4c2f436be31eb" member_type: VOTER last_known_addr { host: "127.20.125.189" port: 44285 } attrs { promote: false } } peers { permanent_uuid: "dece4cc043764f62a3667954979c5562" member_type: NON_VOTER last_known_addr { host: "127.20.125.188" port: 36019 } attrs { promote: true } } }
I20260809 09:32:57.889441 31038 sys_catalog.cc:458] T 00000000000000000000000000000000 P ec1960de545a42ba9ac3e3e9adc99056 [sys.catalog]: This master's current role is: LEADER
I20260809 09:32:57.889948 31055 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P dece4cc043764f62a3667954979c5562: tablet copy: Beginning tablet copy session from remote peer at address 127.20.125.190:37175
I20260809 09:32:57.894935 31117 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 3c029f42802647dea9a4c2f436be31eb [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 3c029f42802647dea9a4c2f436be31eb (127.20.125.189) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "ec1960de545a42ba9ac3e3e9adc99056" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 37175 } } peers { permanent_uuid: "3c029f42802647dea9a4c2f436be31eb" member_type: VOTER last_known_addr { host: "127.20.125.189" port: 44285 } attrs { promote: false } } peers { permanent_uuid: "dece4cc043764f62a3667954979c5562" member_type: NON_VOTER last_known_addr { host: "127.20.125.188" port: 36019 } attrs { promote: true } } }
I20260809 09:32:57.897245 31340 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3c029f42802647dea9a4c2f436be31eb [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ec1960de545a42ba9ac3e3e9adc99056" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "ec1960de545a42ba9ac3e3e9adc99056" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 37175 } } peers { permanent_uuid: "3c029f42802647dea9a4c2f436be31eb" member_type: VOTER last_known_addr { host: "127.20.125.189" port: 44285 } attrs { promote: false } } peers { permanent_uuid: "dece4cc043764f62a3667954979c5562" member_type: NON_VOTER last_known_addr { host: "127.20.125.188" port: 36019 } attrs { promote: true } } }
I20260809 09:32:57.897367 31340 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3c029f42802647dea9a4c2f436be31eb [sys.catalog]: This master's current role is: FOLLOWER
I20260809 09:32:57.900192 31352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:57.890991 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:37175 (local address 127.0.0.1:52698)
0809 09:32:57.891274 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0809 09:32:57.891288 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 09:32:57.891431 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:32:57.891802 (+ 371us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:32:57.891804 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:32:57.892033 (+ 229us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:32:57.892139 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:32:57.892146 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:32:57.893201 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:32:57.893204 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:32:57.894066 (+ 862us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:32:57.894077 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:32:57.894810 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:32:57.894813 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:32:57.894894 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:57.895452 (+ 558us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 09:32:57.895467 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:32:57.895933 (+ 466us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 09:32:57.898079 (+ 2146us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:32:57.898082 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:32:57.898085 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:32:57.898449 (+ 364us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:32:57.898862 (+ 413us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:32:57.898865 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:32:57.898867 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:32:57.899109 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:32:57.899622 (+ 513us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 09:32:57.899626 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 09:32:57.899741 (+ 115us) client_negotiation.cc:769] Sending connection context
0809 09:32:57.899856 (+ 115us) client_negotiation.cc:240] Negotiation successful
0809 09:32:57.900003 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":61,"threads_started":1}
I20260809 09:32:57.900192 30241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:57.890687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52698 (local address 127.20.125.190:37175)
0809 09:32:57.890805 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0809 09:32:57.890810 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 09:32:57.891356 (+ 546us) server_negotiation.cc:407] Connection header received
0809 09:32:57.891538 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:32:57.891542 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:32:57.891589 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:32:57.891640 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:32:57.892276 (+ 636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:57.893053 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:57.894210 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:57.894550 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:57.894633 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:57.896095 (+ 1462us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:32:57.896099 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:32:57.896102 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:32:57.896128 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:32:57.897974 (+ 1846us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:32:57.898617 (+ 643us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:32:57.898623 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:32:57.898627 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:32:57.898748 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:32:57.899226 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:32:57.899229 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:32:57.899231 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:32:57.899514 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:32:57.899618 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0809 09:32:57.899929 (+ 311us) server_negotiation.cc:299] Negotiation successful
0809 09:32:57.900049 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260809 09:32:57.900731 30210 tablet_copy_service.cc:140] P ec1960de545a42ba9ac3e3e9adc99056: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer dece4cc043764f62a3667954979c5562 ({username='oryx', principal='oryx/127.20.125.188@KRBTEST.COM'} at 127.0.0.1:52698)
I20260809 09:32:57.900830 30210 tablet_copy_service.cc:161] P ec1960de545a42ba9ac3e3e9adc99056: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer dece4cc043764f62a3667954979c5562 at {username='oryx', principal='oryx/127.20.125.188@KRBTEST.COM'} at 127.0.0.1:52698: session id = dece4cc043764f62a3667954979c5562-00000000000000000000000000000000
I20260809 09:32:57.901871 30210 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P ec1960de545a42ba9ac3e3e9adc99056: Tablet Copy: opened 0 blocks and 1 log segments
I20260809 09:32:57.903184 31055 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260809 09:32:57.905813 31055 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P dece4cc043764f62a3667954979c5562: tablet copy: Starting download of 0 data blocks...
I20260809 09:32:57.906021 31055 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P dece4cc043764f62a3667954979c5562: tablet copy: Starting download of 1 WAL segments...
I20260809 09:32:57.907672 31055 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P dece4cc043764f62a3667954979c5562: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260809 09:32:57.909488 30210 tablet_copy_service.cc:342] P ec1960de545a42ba9ac3e3e9adc99056: Request end of tablet copy session dece4cc043764f62a3667954979c5562-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.20.125.188@KRBTEST.COM'} at 127.0.0.1:52698
I20260809 09:32:57.909574 30210 tablet_copy_service.cc:433] P ec1960de545a42ba9ac3e3e9adc99056: ending tablet copy session dece4cc043764f62a3667954979c5562-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer dece4cc043764f62a3667954979c5562
I20260809 09:32:57.912662 30687 heartbeater.cc:503] Master 127.20.125.190:37175 was elected leader, sending a full tablet report...
I20260809 09:32:57.917470 31055 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 09:32:57.919955 31353 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:32:57.923686 31356 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:32:57.923898 31358 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:32:57.926254 31055 server_base.cc:1034] running on GCE node
I20260809 09:32:57.926432 31055 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 09:32:57.926823 31055 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 09:32:57.927978 31055 hybrid_clock.cc:648] HybridClock initialized: now 1786267977927955 us; error 42 us; skew 500 ppm
Aug 09 09:32:57 dist-test-slave-7kzw krb5kdc[30113](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786267977, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.125.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 09:32:57.929677 31055 init.cc:377] Logged in from keytab as oryx/127.20.125.188@KRBTEST.COM (short username oryx)
I20260809 09:32:57.930171 31055 webserver.cc:460] Webserver started at http://127.20.125.188:37599/ using document root <none> and password file <none>
I20260809 09:32:57.930368 31055 fs_manager.cc:359] Metadata directory not provided
I20260809 09:32:57.930413 31055 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 09:32:57.931887 31055 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 09:32:57.932971 31366 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 09:32:57.933234 31055 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 09:32:57.933317 31055 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/minicluster-data/master-2/data,/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/minicluster-data/master-2/wal
uuid: "dece4cc043764f62a3667954979c5562"
format_stamp: "Formatted at 2026-08-09 09:32:57 on dist-test-slave-7kzw"
server_key: "a2e68086116e8cf055924a594ccc5357"
server_key_iv: "95a697b4f70c6b73bffc7a60e6c85757"
server_key_version: "encryptionkey@0"
I20260809 09:32:57.933405 31055 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 09:32:57.947259 31055 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 09:32:57.949482 31055 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 09:32:57.954540 31055 rpc_server.cc:307] RPC server started. Bound to: 127.20.125.188:36019
I20260809 09:32:57.954905 31436 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.125.188:36019 every 8 connection(s)
I20260809 09:32:57.955157 31055 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/minicluster-data/master-2/data/info.pb
I20260809 09:32:57.957135 31439 sys_catalog.cc:263] Verifying existing consensus state
I20260809 09:32:57.958228 31439 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dece4cc043764f62a3667954979c5562: Bootstrap starting.
I20260809 09:32:57.970448 31439 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dece4cc043764f62a3667954979c5562: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 09:32:57.970824 31439 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dece4cc043764f62a3667954979c5562: Bootstrap complete.
I20260809 09:32:57.971500 31439 raft_consensus.cc:348] T 00000000000000000000000000000000 P dece4cc043764f62a3667954979c5562 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "ec1960de545a42ba9ac3e3e9adc99056" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 37175 } } peers { permanent_uuid: "3c029f42802647dea9a4c2f436be31eb" member_type: VOTER last_known_addr { host: "127.20.125.189" port: 44285 } attrs { promote: false } } peers { permanent_uuid: "dece4cc043764f62a3667954979c5562" member_type: NON_VOTER last_known_addr { host: "127.20.125.188" port: 36019 } attrs { promote: true } }
I20260809 09:32:57.971704 31439 raft_consensus.cc:729] T 00000000000000000000000000000000 P dece4cc043764f62a3667954979c5562 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: dece4cc043764f62a3667954979c5562, State: Initialized, Role: LEARNER
I20260809 09:32:57.971838 31439 consensus_queue.cc:260] T 00000000000000000000000000000000 P dece4cc043764f62a3667954979c5562 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "ec1960de545a42ba9ac3e3e9adc99056" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 37175 } } peers { permanent_uuid: "3c029f42802647dea9a4c2f436be31eb" member_type: VOTER last_known_addr { host: "127.20.125.189" port: 44285 } attrs { promote: false } } peers { permanent_uuid: "dece4cc043764f62a3667954979c5562" member_type: NON_VOTER last_known_addr { host: "127.20.125.188" port: 36019 } attrs { promote: true } }
I20260809 09:32:57.972234 31442 sys_catalog.cc:455] T 00000000000000000000000000000000 P dece4cc043764f62a3667954979c5562 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "ec1960de545a42ba9ac3e3e9adc99056" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 37175 } } peers { permanent_uuid: "3c029f42802647dea9a4c2f436be31eb" member_type: VOTER last_known_addr { host: "127.20.125.189" port: 44285 } attrs { promote: false } } peers { permanent_uuid: "dece4cc043764f62a3667954979c5562" member_type: NON_VOTER last_known_addr { host: "127.20.125.188" port: 36019 } attrs { promote: true } } }
I20260809 09:32:57.972386 31439 sys_catalog.cc:564] T 00000000000000000000000000000000 P dece4cc043764f62a3667954979c5562 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 09:32:57.972435 31442 sys_catalog.cc:458] T 00000000000000000000000000000000 P dece4cc043764f62a3667954979c5562 [sys.catalog]: This master's current role is: LEARNER
I20260809 09:32:57.976627 31462 catalog_manager.cc:1259] Loaded cluster ID: a876011cffd24bdd84e8602f372dcb24
I20260809 09:32:57.976698 31462 catalog_manager.cc:1552] T 00000000000000000000000000000000 P dece4cc043764f62a3667954979c5562: loading cluster ID for follower catalog manager: success
I20260809 09:32:57.978992 31462 catalog_manager.cc:1574] T 00000000000000000000000000000000 P dece4cc043764f62a3667954979c5562: acquiring CA information for follower catalog manager: success
I20260809 09:32:57.980415 31462 catalog_manager.cc:1602] T 00000000000000000000000000000000 P dece4cc043764f62a3667954979c5562: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260809 09:32:58.007999 31338 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 09:32:58.008289 31338 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 09:32:58.008334 31338 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 09:32:58.012370 31338 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 09:32:58.012432 31338 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 09:32:58.012492 31338 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.125.129
I20260809 09:32:58.017831 31338 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.125.148:36081
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/krb5kdc/oryx_127.20.125.129.keytab
--principal=oryx/127.20.125.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.125.129:42545
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.125.129
--webserver_port=35525
--tserver_master_addrs=127.20.125.190:37175,127.20.125.189:44285,127.20.125.188:36019
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31338
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.125.129
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 08:38:55 UTC on ccycloud.quasar-njjleu.root.comops.site
build id 4575935
I20260809 09:32:58.018952 31338 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 09:32:58.019972 31338 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 09:32:58.029817 31477 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:32:58.029817 31475 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:32:58.030037 31338 server_base.cc:1034] running on GCE node
W20260809 09:32:58.029816 31474 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:32:58.030624 31338 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 09:32:58.031292 31338 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 09:32:58.032485 31338 hybrid_clock.cc:648] HybridClock initialized: now 1786267978032468 us; error 26 us; skew 500 ppm
Aug 09 09:32:58 dist-test-slave-7kzw krb5kdc[30113](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786267978, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.125.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 09:32:58.035394 31338 init.cc:377] Logged in from keytab as oryx/127.20.125.129@KRBTEST.COM (short username oryx)
I20260809 09:32:58.036311 31338 webserver.cc:460] Webserver started at http://127.20.125.129:35525/ using document root <none> and password file <none>
I20260809 09:32:58.036885 31338 fs_manager.cc:359] Metadata directory not provided
I20260809 09:32:58.036998 31338 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 09:32:58.040957 31338 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.001s
I20260809 09:32:58.043077 31489 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 09:32:58.044348 31338 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 09:32:58.044489 31338 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/minicluster-data/ts-0/data,/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/minicluster-data/ts-0/wal
uuid: "143f334e51d54d17911a2d2ebb0b6554"
format_stamp: "Formatted at 2026-08-09 09:32:56 on dist-test-slave-7kzw"
server_key: "1e2e593bc163a00e9387565016b42f77"
server_key_iv: "4ccac51aca026c965bfaae05b91b9a72"
server_key_version: "encryptionkey@0"
I20260809 09:32:58.045043 31338 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 09:32:58.058064 31338 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 09:32:58.061165 31338 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 09:32:58.061386 31338 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 09:32:58.062000 31338 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 09:32:58.062891 31338 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 09:32:58.062968 31338 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 09:32:58.063040 31338 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 09:32:58.063094 31338 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 09:32:58.075539 31338 rpc_server.cc:307] RPC server started. Bound to: 127.20.125.129:42545
I20260809 09:32:58.075573 31656 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.125.129:42545 every 8 connection(s)
I20260809 09:32:58.076592 31338 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/minicluster-data/ts-0/data/info.pb
I20260809 09:32:58.079001 20982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu as pid 31338
I20260809 09:32:58.079200 20982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu with pid 30491
Aug 09 09:32:58 dist-test-slave-7kzw krb5kdc[30113](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786267978, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.125.129@KRBTEST.COM for oryx/127.20.125.189@KRBTEST.COM
Aug 09 09:32:58 dist-test-slave-7kzw krb5kdc[30113](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786267978, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.125.129@KRBTEST.COM for oryx/127.20.125.190@KRBTEST.COM
Aug 09 09:32:58 dist-test-slave-7kzw krb5kdc[30113](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786267978, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.125.129@KRBTEST.COM for oryx/127.20.125.188@KRBTEST.COM
I20260809 09:32:58.094831 20982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
/tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.125.130:44297
--local_ip_for_outbound_sockets=127.20.125.130
--tserver_master_addrs=127.20.125.190:37175
--webserver_port=33213
--webserver_interface=127.20.125.130
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/krb5kdc/oryx_127.20.125.130.keytab
--principal=oryx/127.20.125.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.125.148:36081
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.20.125.190:37175,127.20.125.189:44285
--tserver_master_addrs=127.20.125.190:37175,127.20.125.189:44285,127.20.125.188:36019 with env {KRB5CCNAME=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 09:32:58.096823 31304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:58.079267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.129:44489 (local address 127.20.125.189:44285)
0809 09:32:58.079606 (+ 339us) server_negotiation.cc:206] Beginning negotiation
0809 09:32:58.079610 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 09:32:58.080488 (+ 878us) server_negotiation.cc:407] Connection header received
0809 09:32:58.081826 (+ 1338us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:32:58.081832 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:32:58.081898 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:32:58.081963 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:32:58.084042 (+ 2079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:58.085135 (+ 1093us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:58.086796 (+ 1661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:58.087157 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:58.087251 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:58.091841 (+ 4590us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:32:58.091848 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:32:58.091853 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:32:58.091886 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:32:58.093956 (+ 2070us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:32:58.094727 (+ 771us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:32:58.094733 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:32:58.094739 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:32:58.094840 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:32:58.095333 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:32:58.095339 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:32:58.095344 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:32:58.095609 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:32:58.095712 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0809 09:32:58.096482 (+ 770us) server_negotiation.cc:299] Negotiation successful
0809 09:32:58.096621 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":242}
I20260809 09:32:58.097314 30241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:58.078612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.129:41547 (local address 127.20.125.190:37175)
0809 09:32:58.079482 (+ 870us) server_negotiation.cc:206] Beginning negotiation
0809 09:32:58.079488 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 09:32:58.082816 (+ 3328us) server_negotiation.cc:407] Connection header received
0809 09:32:58.083040 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:32:58.083047 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:32:58.083128 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:32:58.083230 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:32:58.084000 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:58.084793 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:58.086829 (+ 2036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:58.087316 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:58.087424 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:58.091827 (+ 4403us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:32:58.091834 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:32:58.091841 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:32:58.091879 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:32:58.095021 (+ 3142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:32:58.095636 (+ 615us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:32:58.095643 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:32:58.095649 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:32:58.095756 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:32:58.096162 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:32:58.096169 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:32:58.096174 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:32:58.096450 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:32:58.096555 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0809 09:32:58.097003 (+ 448us) server_negotiation.cc:299] Negotiation successful
0809 09:32:58.097137 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":765}
I20260809 09:32:58.098384 31669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:58.079832 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.129:52515 (local address 127.20.125.188:36019)
0809 09:32:58.085631 (+ 5799us) server_negotiation.cc:206] Beginning negotiation
0809 09:32:58.085637 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 09:32:58.085655 (+ 18us) server_negotiation.cc:407] Connection header received
0809 09:32:58.085703 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:32:58.085707 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:32:58.085763 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:32:58.086419 (+ 656us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:32:58.086645 (+ 226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:58.087820 (+ 1175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:58.089625 (+ 1805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:58.090105 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:58.090243 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:58.092953 (+ 2710us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:32:58.092957 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:32:58.092960 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:32:58.092995 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:32:58.095302 (+ 2307us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:32:58.095852 (+ 550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:32:58.095855 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:32:58.095857 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:32:58.095963 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:32:58.096311 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:32:58.096315 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:32:58.096317 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:32:58.096574 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:32:58.097988 (+ 1414us) server_negotiation.cc:1035] Waiting for connection context
0809 09:32:58.098080 (+ 92us) server_negotiation.cc:299] Negotiation successful
0809 09:32:58.098217 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5708,"thread_start_us":1946,"threads_started":1}
I20260809 09:32:58.098585 31665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:58.079039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:37175 (local address 127.20.125.129:41547)
0809 09:32:58.082712 (+ 3673us) negotiation.cc:107] Waiting for socket to connect
0809 09:32:58.082729 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 09:32:58.082897 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:32:58.083390 (+ 493us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:32:58.083397 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:32:58.083666 (+ 269us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:32:58.083849 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:32:58.083860 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:32:58.084970 (+ 1110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:32:58.084976 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:32:58.086665 (+ 1689us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:32:58.086682 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:32:58.087633 (+ 951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:32:58.087640 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:32:58.087781 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:58.088621 (+ 840us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 09:32:58.088645 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:32:58.091635 (+ 2990us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 09:32:58.095193 (+ 3558us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:32:58.095199 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:32:58.095205 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:32:58.095500 (+ 295us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:32:58.095909 (+ 409us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:32:58.095915 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:32:58.095921 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:32:58.096027 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:32:58.096631 (+ 604us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 09:32:58.096637 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 09:32:58.096756 (+ 119us) client_negotiation.cc:769] Sending connection context
0809 09:32:58.096929 (+ 173us) client_negotiation.cc:240] Negotiation successful
0809 09:32:58.097343 (+ 414us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3581,"mutex_wait_us":1052,"spinlock_wait_cycles":8064,"thread_start_us":1655,"threads_started":1}
I20260809 09:32:58.098713 31670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:58.079022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.188:36019 (local address 127.20.125.129:52515)
0809 09:32:58.082134 (+ 3112us) negotiation.cc:107] Waiting for socket to connect
0809 09:32:58.082152 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 09:32:58.082247 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:32:58.085931 (+ 3684us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:32:58.085934 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:32:58.086156 (+ 222us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:32:58.086282 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:32:58.086290 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:32:58.088599 (+ 2309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:32:58.088605 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:32:58.089484 (+ 879us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:32:58.089500 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:32:58.090279 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:32:58.090286 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:32:58.090412 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:58.091112 (+ 700us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 09:32:58.091129 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:32:58.092765 (+ 1636us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 09:32:58.095427 (+ 2662us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:32:58.095431 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:32:58.095432 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:32:58.095716 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:32:58.096085 (+ 369us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:32:58.096088 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:32:58.096090 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:32:58.096181 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:32:58.096705 (+ 524us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 09:32:58.096709 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 09:32:58.096803 (+ 94us) client_negotiation.cc:769] Sending connection context
0809 09:32:58.096929 (+ 126us) client_negotiation.cc:240] Negotiation successful
0809 09:32:58.097420 (+ 491us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3009,"mutex_wait_us":187,"thread_start_us":190,"threads_started":1}
I20260809 09:32:58.098914 31664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:58.079022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.189:44285 (local address 127.20.125.129:44489)
0809 09:32:58.080324 (+ 1302us) negotiation.cc:107] Waiting for socket to connect
0809 09:32:58.080381 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0809 09:32:58.081561 (+ 1180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:32:58.082434 (+ 873us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:32:58.082447 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:32:58.083081 (+ 634us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:32:58.083802 (+ 721us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:32:58.083830 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:32:58.085328 (+ 1498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:32:58.085334 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:32:58.086645 (+ 1311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:32:58.086655 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:32:58.087490 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:32:58.087494 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:32:58.087699 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:58.088497 (+ 798us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 09:32:58.088524 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:32:58.091603 (+ 3079us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 09:32:58.094114 (+ 2511us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:32:58.094121 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:32:58.094139 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:32:58.094584 (+ 445us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:32:58.094978 (+ 394us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:32:58.094982 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:32:58.094985 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:32:58.095201 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:32:58.095789 (+ 588us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 09:32:58.095799 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 09:32:58.096155 (+ 356us) client_negotiation.cc:769] Sending connection context
0809 09:32:58.096977 (+ 822us) client_negotiation.cc:240] Negotiation successful
0809 09:32:58.097706 (+ 729us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1086,"mutex_wait_us":497,"thread_start_us":143,"threads_started":1}
I20260809 09:32:58.099947 31658 heartbeater.cc:347] Connected to a master server at 127.20.125.190:37175
I20260809 09:32:58.099968 31660 heartbeater.cc:347] Connected to a master server at 127.20.125.189:44285
I20260809 09:32:58.100253 31660 heartbeater.cc:464] Registering TS with master...
I20260809 09:32:58.100711 31657 heartbeater.cc:347] Connected to a master server at 127.20.125.188:36019
I20260809 09:32:58.100831 31657 heartbeater.cc:464] Registering TS with master...
I20260809 09:32:58.100981 31660 heartbeater.cc:511] Master 127.20.125.189:44285 requested a full tablet report, sending...
I20260809 09:32:58.101066 31657 heartbeater.cc:511] Master 127.20.125.188:36019 requested a full tablet report, sending...
I20260809 09:32:58.100253 31658 heartbeater.cc:464] Registering TS with master...
I20260809 09:32:58.101832 31658 heartbeater.cc:511] Master 127.20.125.190:37175 requested a full tablet report, sending...
I20260809 09:32:58.102672 31112 ts_manager.cc:194] Registered new tserver with Master: 143f334e51d54d17911a2d2ebb0b6554 (127.20.125.129:42545)
I20260809 09:32:58.102671 31389 ts_manager.cc:194] Registered new tserver with Master: 143f334e51d54d17911a2d2ebb0b6554 (127.20.125.129:42545)
I20260809 09:32:58.102860 30181 ts_manager.cc:194] Re-registered known tserver with Master: 143f334e51d54d17911a2d2ebb0b6554 (127.20.125.129:42545)
I20260809 09:32:58.103744 30181 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.20.125.129@KRBTEST.COM'} at 127.20.125.129:41547
I20260809 09:32:58.169121 30888 heartbeater.cc:503] Master 127.20.125.190:37175 was elected leader, sending a full tablet report...
W20260809 09:32:58.224581 31671 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 09:32:58.224872 31671 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 09:32:58.224920 31671 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 09:32:58.229135 31671 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 09:32:58.229203 31671 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 09:32:58.229295 31671 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.125.130
I20260809 09:32:58.234915 31671 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.125.148:36081
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/krb5kdc/oryx_127.20.125.130.keytab
--principal=oryx/127.20.125.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.125.130:44297
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.125.130
--webserver_port=33213
--tserver_master_addrs=127.20.125.190:37175,127.20.125.189:44285,127.20.125.188:36019
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31671
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.125.130
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 08:38:55 UTC on ccycloud.quasar-njjleu.root.comops.site
build id 4575935
I20260809 09:32:58.236004 31671 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 09:32:58.237149 31671 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 09:32:58.247540 31685 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:32:58.247529 31677 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:32:58.247617 31682 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:32:58.247884 31671 server_base.cc:1034] running on GCE node
I20260809 09:32:58.248437 31671 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 09:32:58.249217 31671 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 09:32:58.250403 31671 hybrid_clock.cc:648] HybridClock initialized: now 1786267978250347 us; error 88 us; skew 500 ppm
Aug 09 09:32:58 dist-test-slave-7kzw krb5kdc[30113](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786267978, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.125.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 09:32:58.254249 31671 init.cc:377] Logged in from keytab as oryx/127.20.125.130@KRBTEST.COM (short username oryx)
I20260809 09:32:58.255386 31671 webserver.cc:460] Webserver started at http://127.20.125.130:33213/ using document root <none> and password file <none>
I20260809 09:32:58.256052 31671 fs_manager.cc:359] Metadata directory not provided
I20260809 09:32:58.256138 31671 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 09:32:58.260459 31671 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260809 09:32:58.262569 31694 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 09:32:58.263928 31671 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 09:32:58.264107 31671 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/minicluster-data/ts-1/data,/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/minicluster-data/ts-1/wal
uuid: "8bbe397caa0242579d9743094319a762"
format_stamp: "Formatted at 2026-08-09 09:32:56 on dist-test-slave-7kzw"
server_key: "9f42e2e53fdd35eef20409db80a49ef8"
server_key_iv: "c0601ac9cb68cab291ed792dd1cf81e3"
server_key_version: "encryptionkey@0"
I20260809 09:32:58.264675 31671 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 09:32:58.277269 31671 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 09:32:58.280433 31671 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 09:32:58.280650 31671 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 09:32:58.281277 31671 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 09:32:58.282198 31671 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 09:32:58.282250 31671 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 09:32:58.282294 31671 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 09:32:58.282351 31671 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 09:32:58.293401 31671 rpc_server.cc:307] RPC server started. Bound to: 127.20.125.130:44297
I20260809 09:32:58.293431 31860 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.125.130:44297 every 8 connection(s)
I20260809 09:32:58.294523 31671 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/minicluster-data/ts-1/data/info.pb
I20260809 09:32:58.298913 20982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu as pid 31671
I20260809 09:32:58.299139 20982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu with pid 30697
Aug 09 09:32:58 dist-test-slave-7kzw krb5kdc[30113](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786267978, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.125.130@KRBTEST.COM for oryx/127.20.125.189@KRBTEST.COM
I20260809 09:32:58.331851 31304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:58.296457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.130:56655 (local address 127.20.125.189:44285)
0809 09:32:58.296598 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0809 09:32:58.296603 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 09:32:58.298192 (+ 1589us) server_negotiation.cc:407] Connection header received
0809 09:32:58.301433 (+ 3241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:32:58.301439 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:32:58.301515 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:32:58.301623 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:32:58.318118 (+ 16495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:58.319314 (+ 1196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:58.321986 (+ 2672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:58.322451 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:58.322562 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:58.327028 (+ 4466us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:32:58.327036 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:32:58.327043 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:32:58.327085 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:32:58.328967 (+ 1882us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:32:58.329727 (+ 760us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:32:58.329731 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:32:58.329733 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:32:58.329832 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:32:58.330284 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:32:58.330288 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:32:58.330290 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:32:58.330547 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:32:58.330637 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0809 09:32:58.331418 (+ 781us) server_negotiation.cc:299] Negotiation successful
0809 09:32:58.331616 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260809 09:32:58.333804 31867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:58.297067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.189:44285 (local address 127.20.125.130:56655)
0809 09:32:58.298107 (+ 1040us) negotiation.cc:107] Waiting for socket to connect
0809 09:32:58.298121 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 09:32:58.298772 (+ 651us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:32:58.316522 (+ 17750us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:32:58.316537 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:32:58.317135 (+ 598us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:32:58.317853 (+ 718us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:32:58.317875 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:32:58.319555 (+ 1680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:32:58.319563 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:32:58.321778 (+ 2215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:32:58.321796 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:32:58.322793 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:32:58.322799 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:32:58.323014 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:58.323929 (+ 915us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 09:32:58.323965 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:32:58.326765 (+ 2800us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 09:32:58.329089 (+ 2324us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:32:58.329097 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:32:58.329116 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:32:58.329591 (+ 475us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:32:58.329942 (+ 351us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:32:58.329945 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:32:58.329947 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:32:58.330142 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:32:58.330713 (+ 571us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 09:32:58.330723 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 09:32:58.331109 (+ 386us) client_negotiation.cc:769] Sending connection context
0809 09:32:58.332100 (+ 991us) client_negotiation.cc:240] Negotiation successful
0809 09:32:58.332500 (+ 400us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":938,"mutex_wait_us":55,"thread_start_us":202,"threads_started":1}
Aug 09 09:32:58 dist-test-slave-7kzw krb5kdc[30113](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786267978, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.125.130@KRBTEST.COM for oryx/127.20.125.188@KRBTEST.COM
Aug 09 09:32:58 dist-test-slave-7kzw krb5kdc[30113](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786267978, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.125.130@KRBTEST.COM for oryx/127.20.125.190@KRBTEST.COM
I20260809 09:32:58.335961 31864 heartbeater.cc:347] Connected to a master server at 127.20.125.189:44285
I20260809 09:32:58.336258 31864 heartbeater.cc:464] Registering TS with master...
I20260809 09:32:58.336898 31864 heartbeater.cc:511] Master 127.20.125.189:44285 requested a full tablet report, sending...
I20260809 09:32:58.337420 20982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
/tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.125.131:44279
--local_ip_for_outbound_sockets=127.20.125.131
--tserver_master_addrs=127.20.125.190:37175
--webserver_port=38165
--webserver_interface=127.20.125.131
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/krb5kdc/oryx_127.20.125.131.keytab
--principal=oryx/127.20.125.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.125.148:36081
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.20.125.190:37175,127.20.125.189:44285
--tserver_master_addrs=127.20.125.190:37175,127.20.125.189:44285,127.20.125.188:36019 with env {KRB5CCNAME=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 09:32:58.338294 31112 ts_manager.cc:194] Registered new tserver with Master: 8bbe397caa0242579d9743094319a762 (127.20.125.130:44297)
I20260809 09:32:58.339247 31669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:58.297016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.130:35809 (local address 127.20.125.188:36019)
0809 09:32:58.297145 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0809 09:32:58.297150 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 09:32:58.297726 (+ 576us) server_negotiation.cc:407] Connection header received
0809 09:32:58.299013 (+ 1287us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:32:58.299017 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:32:58.299085 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:32:58.299187 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:32:58.329814 (+ 30627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:58.330627 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:58.331798 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:58.332138 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:58.332222 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:58.334886 (+ 2664us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:32:58.334890 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:32:58.334893 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:32:58.334923 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:32:58.336607 (+ 1684us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:32:58.337175 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:32:58.337189 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:32:58.337192 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:32:58.337287 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:32:58.338066 (+ 779us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:32:58.338072 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:32:58.338077 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:32:58.338352 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:32:58.338444 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0809 09:32:58.338849 (+ 405us) server_negotiation.cc:299] Negotiation successful
0809 09:32:58.339007 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260809 09:32:58.340070 31866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:58.297059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.188:36019 (local address 127.20.125.130:35809)
0809 09:32:58.297577 (+ 518us) negotiation.cc:107] Waiting for socket to connect
0809 09:32:58.297628 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0809 09:32:58.298772 (+ 1144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:32:58.329134 (+ 30362us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:32:58.329137 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:32:58.329474 (+ 337us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:32:58.329631 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:32:58.329641 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:32:58.330792 (+ 1151us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:32:58.330795 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:32:58.331631 (+ 836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:32:58.331641 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:32:58.332381 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:32:58.332384 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:32:58.332532 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:58.333065 (+ 533us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 09:32:58.333080 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:32:58.334728 (+ 1648us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 09:32:58.336739 (+ 2011us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:32:58.336742 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:32:58.336744 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:32:58.337030 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:32:58.337406 (+ 376us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:32:58.337409 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:32:58.337411 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:32:58.337946 (+ 535us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:32:58.338532 (+ 586us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 09:32:58.338537 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 09:32:58.338647 (+ 110us) client_negotiation.cc:769] Sending connection context
0809 09:32:58.339460 (+ 813us) client_negotiation.cc:240] Negotiation successful
0809 09:32:58.339606 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":309,"mutex_wait_us":29,"thread_start_us":135,"threads_started":1}
I20260809 09:32:58.340683 31862 heartbeater.cc:347] Connected to a master server at 127.20.125.188:36019
I20260809 09:32:58.340782 31862 heartbeater.cc:464] Registering TS with master...
I20260809 09:32:58.340968 31862 heartbeater.cc:511] Master 127.20.125.188:36019 requested a full tablet report, sending...
I20260809 09:32:58.341254 30241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:58.296548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.130:33257 (local address 127.20.125.190:37175)
0809 09:32:58.296659 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0809 09:32:58.296664 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 09:32:58.298715 (+ 2051us) server_negotiation.cc:407] Connection header received
0809 09:32:58.299165 (+ 450us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:32:58.299169 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:32:58.299235 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:32:58.299309 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:32:58.329685 (+ 30376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:58.330635 (+ 950us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:58.332678 (+ 2043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:58.333086 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:58.333186 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:58.337398 (+ 4212us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:32:58.337402 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:32:58.337405 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:32:58.337435 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:32:58.339225 (+ 1790us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:32:58.339672 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:32:58.339678 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:32:58.339680 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:32:58.339786 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:32:58.340150 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:32:58.340155 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:32:58.340158 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:32:58.340416 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:32:58.340518 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0809 09:32:58.340912 (+ 394us) server_negotiation.cc:299] Negotiation successful
0809 09:32:58.341060 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260809 09:32:58.341702 31868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:58.297059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:37175 (local address 127.20.125.130:33257)
0809 09:32:58.298380 (+ 1321us) negotiation.cc:107] Waiting for socket to connect
0809 09:32:58.298393 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 09:32:58.299064 (+ 671us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:32:58.328981 (+ 29917us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:32:58.328988 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:32:58.329312 (+ 324us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:32:58.329490 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:32:58.329502 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:32:58.331077 (+ 1575us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:32:58.331083 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:32:58.332514 (+ 1431us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:32:58.332529 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:32:58.333452 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:32:58.333459 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:32:58.333595 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:58.334563 (+ 968us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 09:32:58.334583 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:32:58.337224 (+ 2641us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 09:32:58.339340 (+ 2116us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:32:58.339343 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:32:58.339345 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:32:58.339544 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:32:58.339904 (+ 360us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:32:58.339909 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:32:58.339910 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:32:58.340023 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:32:58.340570 (+ 547us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 09:32:58.340574 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 09:32:58.340692 (+ 118us) client_negotiation.cc:769] Sending connection context
0809 09:32:58.341415 (+ 723us) client_negotiation.cc:240] Negotiation successful
0809 09:32:58.341572 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1237,"mutex_wait_us":165,"spinlock_wait_cycles":44928,"thread_start_us":203,"threads_started":1}
I20260809 09:32:58.341799 31389 ts_manager.cc:194] Registered new tserver with Master: 8bbe397caa0242579d9743094319a762 (127.20.125.130:44297)
I20260809 09:32:58.342321 31863 heartbeater.cc:347] Connected to a master server at 127.20.125.190:37175
I20260809 09:32:58.342415 31863 heartbeater.cc:464] Registering TS with master...
I20260809 09:32:58.342633 31863 heartbeater.cc:511] Master 127.20.125.190:37175 requested a full tablet report, sending...
I20260809 09:32:58.343313 30181 ts_manager.cc:194] Re-registered known tserver with Master: 8bbe397caa0242579d9743094319a762 (127.20.125.130:44297)
I20260809 09:32:58.343880 30181 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.20.125.130@KRBTEST.COM'} at 127.20.125.130:33257
I20260809 09:32:58.362356 31032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:58.358555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.188:36019 (local address 127.0.0.1:37648)
0809 09:32:58.358706 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0809 09:32:58.358731 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0809 09:32:58.358821 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:32:58.359171 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:32:58.359174 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:32:58.359196 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 09:32:58.359285 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:32:58.359290 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:32:58.360335 (+ 1045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:32:58.360338 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:32:58.361143 (+ 805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:32:58.361154 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:32:58.362023 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:32:58.362026 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:32:58.362097 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:58.362118 (+ 21us) client_negotiation.cc:769] Sending connection context
0809 09:32:58.362173 (+ 55us) client_negotiation.cc:240] Negotiation successful
0809 09:32:58.362236 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51}
I20260809 09:32:58.362414 31669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:58.358614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37648 (local address 127.20.125.188:36019)
0809 09:32:58.358754 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0809 09:32:58.358761 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 09:32:58.358784 (+ 23us) server_negotiation.cc:407] Connection header received
0809 09:32:58.358930 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:32:58.358933 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:32:58.358998 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:32:58.359111 (+ 113us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 09:32:58.359400 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:58.360214 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:58.361267 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:58.361897 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:58.362034 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:58.362172 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0809 09:32:58.362220 (+ 48us) server_negotiation.cc:299] Negotiation successful
0809 09:32:58.362284 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260809 09:32:58.363149 31400 raft_consensus.cc:1203] T 00000000000000000000000000000000 P dece4cc043764f62a3667954979c5562 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260809 09:32:58.363588 31442 sys_catalog.cc:455] T 00000000000000000000000000000000 P dece4cc043764f62a3667954979c5562 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ec1960de545a42ba9ac3e3e9adc99056. Latest consensus state: current_term: 1 leader_uuid: "ec1960de545a42ba9ac3e3e9adc99056" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "ec1960de545a42ba9ac3e3e9adc99056" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 37175 } } peers { permanent_uuid: "3c029f42802647dea9a4c2f436be31eb" member_type: VOTER last_known_addr { host: "127.20.125.189" port: 44285 } attrs { promote: false } } peers { permanent_uuid: "dece4cc043764f62a3667954979c5562" member_type: NON_VOTER last_known_addr { host: "127.20.125.188" port: 36019 } attrs { promote: true } } }
I20260809 09:32:58.363732 31442 sys_catalog.cc:458] T 00000000000000000000000000000000 P dece4cc043764f62a3667954979c5562 [sys.catalog]: This master's current role is: LEARNER
W20260809 09:32:58.463883 31870 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 09:32:58.464215 31870 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 09:32:58.464295 31870 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 09:32:58.468607 31870 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 09:32:58.468699 31870 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 09:32:58.468792 31870 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.125.131
I20260809 09:32:58.474761 31870 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.125.148:36081
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/krb5kdc/oryx_127.20.125.131.keytab
--principal=oryx/127.20.125.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.125.131:44279
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.125.131
--webserver_port=38165
--tserver_master_addrs=127.20.125.190:37175,127.20.125.189:44285,127.20.125.188:36019
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31870
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.125.131
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 08:38:55 UTC on ccycloud.quasar-njjleu.root.comops.site
build id 4575935
I20260809 09:32:58.476022 31870 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 09:32:58.477175 31870 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 09:32:58.487207 31878 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:32:58.487212 31880 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:32:58.487228 31882 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:32:58.487936 31870 server_base.cc:1034] running on GCE node
I20260809 09:32:58.488405 31870 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 09:32:58.489143 31870 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 09:32:58.490368 31870 hybrid_clock.cc:648] HybridClock initialized: now 1786267978490338 us; error 41 us; skew 500 ppm
Aug 09 09:32:58 dist-test-slave-7kzw krb5kdc[30113](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786267978, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.125.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 09:32:58.493484 31870 init.cc:377] Logged in from keytab as oryx/127.20.125.131@KRBTEST.COM (short username oryx)
I20260809 09:32:58.494439 31870 webserver.cc:460] Webserver started at http://127.20.125.131:38165/ using document root <none> and password file <none>
I20260809 09:32:58.495141 31870 fs_manager.cc:359] Metadata directory not provided
I20260809 09:32:58.495230 31870 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 09:32:58.499938 31870 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.002s
I20260809 09:32:58.502210 31897 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 09:32:58.503566 31870 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 09:32:58.503698 31870 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/minicluster-data/ts-2/data,/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/minicluster-data/ts-2/wal
uuid: "825ed73d552f4f8cbce972a3dc70ccd8"
format_stamp: "Formatted at 2026-08-09 09:32:57 on dist-test-slave-7kzw"
server_key: "0b550ab544b5e99f67e8b7905baf5fc7"
server_key_iv: "24adfb2f986e24968c618f4dc82821d2"
server_key_version: "encryptionkey@0"
I20260809 09:32:58.504288 31870 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 09:32:58.516244 31870 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 09:32:58.519959 31870 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 09:32:58.520195 31870 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 09:32:58.520879 31870 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 09:32:58.521816 31870 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 09:32:58.521867 31870 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 09:32:58.521943 31870 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 09:32:58.521978 31870 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 09:32:58.532771 31870 rpc_server.cc:307] RPC server started. Bound to: 127.20.125.131:44279
I20260809 09:32:58.532796 32060 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.125.131:44279 every 8 connection(s)
I20260809 09:32:58.534065 31870 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/minicluster-data/ts-2/data/info.pb
I20260809 09:32:58.537240 20982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu as pid 31870
Aug 09 09:32:58 dist-test-slave-7kzw krb5kdc[30113](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786267978, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.125.131@KRBTEST.COM for oryx/127.20.125.188@KRBTEST.COM
Aug 09 09:32:58 dist-test-slave-7kzw krb5kdc[30113](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786267978, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.125.131@KRBTEST.COM for oryx/127.20.125.189@KRBTEST.COM
Aug 09 09:32:58 dist-test-slave-7kzw krb5kdc[30113](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786267978, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.125.131@KRBTEST.COM for oryx/127.20.125.190@KRBTEST.COM
I20260809 09:32:58.553745 32090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:58.539789 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52710 (local address 127.20.125.190:37175)
0809 09:32:58.541131 (+ 1342us) server_negotiation.cc:206] Beginning negotiation
0809 09:32:58.541159 (+ 28us) server_negotiation.cc:399] Waiting for connection header
0809 09:32:58.541174 (+ 15us) server_negotiation.cc:407] Connection header received
0809 09:32:58.541216 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:32:58.541219 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:32:58.541267 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:32:58.541368 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:32:58.542270 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:58.543210 (+ 940us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:58.544784 (+ 1574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:58.545198 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:58.545299 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:58.547579 (+ 2280us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:32:58.547585 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:32:58.547589 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:32:58.547625 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:32:58.550925 (+ 3300us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:32:58.551656 (+ 731us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:32:58.551663 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:32:58.551667 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:32:58.551780 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:32:58.552154 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:32:58.552158 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:32:58.552160 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:32:58.552419 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:32:58.552586 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0809 09:32:58.553312 (+ 726us) server_negotiation.cc:299] Negotiation successful
0809 09:32:58.553487 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1258,"thread_start_us":127,"threads_started":1}
I20260809 09:32:58.555914 32091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:58.539942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37650 (local address 127.20.125.188:36019)
0809 09:32:58.540200 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0809 09:32:58.540203 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 09:32:58.540881 (+ 678us) server_negotiation.cc:407] Connection header received
0809 09:32:58.540988 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:32:58.540990 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:32:58.541030 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:32:58.541140 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:32:58.542294 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:58.543197 (+ 903us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:58.544784 (+ 1587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:58.545440 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:58.545540 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:58.548265 (+ 2725us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:32:58.548271 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:32:58.548277 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:32:58.548306 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:32:58.551145 (+ 2839us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:32:58.552369 (+ 1224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:32:58.552374 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:32:58.552377 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:32:58.552478 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:32:58.552852 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:32:58.552856 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:32:58.552860 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:32:58.553112 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:32:58.554965 (+ 1853us) server_negotiation.cc:1035] Waiting for connection context
0809 09:32:58.555052 (+ 87us) server_negotiation.cc:299] Negotiation successful
0809 09:32:58.555255 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":183,"thread_start_us":112,"threads_started":1}
I20260809 09:32:58.555950 32092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:58.539966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49634 (local address 127.20.125.189:44285)
0809 09:32:58.540580 (+ 614us) server_negotiation.cc:206] Beginning negotiation
0809 09:32:58.540584 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 09:32:58.545367 (+ 4783us) server_negotiation.cc:407] Connection header received
0809 09:32:58.545531 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:32:58.545534 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:32:58.545578 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:32:58.545635 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:32:58.547190 (+ 1555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:58.547932 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:58.549262 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:58.549619 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:58.549705 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:58.551699 (+ 1994us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:32:58.551703 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:32:58.551705 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:32:58.551730 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:32:58.553825 (+ 2095us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:32:58.554285 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:32:58.554290 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:32:58.554292 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:32:58.554412 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:32:58.554802 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:32:58.554806 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:32:58.554808 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:32:58.555045 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:32:58.555166 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0809 09:32:58.555611 (+ 445us) server_negotiation.cc:299] Negotiation successful
0809 09:32:58.555754 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":541,"thread_start_us":103,"threads_started":1}
I20260809 09:32:58.558810 31669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:58.536209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.131:35805 (local address 127.20.125.188:36019)
0809 09:32:58.536372 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0809 09:32:58.536377 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 09:32:58.539102 (+ 2725us) server_negotiation.cc:407] Connection header received
0809 09:32:58.539970 (+ 868us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:32:58.539973 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:32:58.540035 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:32:58.540100 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:32:58.542010 (+ 1910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:58.542798 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:58.544714 (+ 1916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:58.545080 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:58.545184 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:58.552481 (+ 7297us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:32:58.552489 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:32:58.552492 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:32:58.552520 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:32:58.554592 (+ 2072us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:32:58.555654 (+ 1062us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:32:58.555659 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:32:58.555662 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:32:58.555762 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:32:58.556263 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:32:58.556269 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:32:58.556274 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:32:58.556547 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:32:58.556671 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0809 09:32:58.558548 (+ 1877us) server_negotiation.cc:299] Negotiation successful
0809 09:32:58.558667 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":19,"server-negotiator.queue_time_us":70}
I20260809 09:32:58.557476 30181 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52710:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 09:32:58.560829 32076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:58.536639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.188:36019 (local address 127.20.125.131:35805)
0809 09:32:58.538891 (+ 2252us) negotiation.cc:107] Waiting for socket to connect
0809 09:32:58.538904 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 09:32:58.539713 (+ 809us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:32:58.540533 (+ 820us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:32:58.540568 (+ 35us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:32:58.541144 (+ 576us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:32:58.541854 (+ 710us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:32:58.541875 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:32:58.542971 (+ 1096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:32:58.542974 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:32:58.544569 (+ 1595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:32:58.544580 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:32:58.546437 (+ 1857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:32:58.546442 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:32:58.546656 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:58.549094 (+ 2438us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 09:32:58.549118 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:32:58.551874 (+ 2756us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 09:32:58.554764 (+ 2890us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:32:58.554773 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:32:58.554794 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:32:58.555213 (+ 419us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:32:58.555883 (+ 670us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:32:58.555885 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:32:58.555887 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:32:58.556123 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:32:58.557911 (+ 1788us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 09:32:58.557918 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 09:32:58.558243 (+ 325us) client_negotiation.cc:769] Sending connection context
0809 09:32:58.559169 (+ 926us) client_negotiation.cc:240] Negotiation successful
0809 09:32:58.559529 (+ 360us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2175,"mutex_wait_us":52,"spinlock_wait_cycles":32512,"thread_start_us":220,"threads_started":1}
I20260809 09:32:58.561043 31304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:58.536259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.131:43511 (local address 127.20.125.189:44285)
0809 09:32:58.536387 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0809 09:32:58.536391 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 09:32:58.539569 (+ 3178us) server_negotiation.cc:407] Connection header received
0809 09:32:58.541888 (+ 2319us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:32:58.541892 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:32:58.541947 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:32:58.542003 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:32:58.544200 (+ 2197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:58.544945 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:58.547832 (+ 2887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:58.548140 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:58.548208 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:58.556520 (+ 8312us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:32:58.556524 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:32:58.556527 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:32:58.556553 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:32:58.558659 (+ 2106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:32:58.559412 (+ 753us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:32:58.559419 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:32:58.559421 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:32:58.559568 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:32:58.559965 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:32:58.559969 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:32:58.559971 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:32:58.560212 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:32:58.560305 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0809 09:32:58.560712 (+ 407us) server_negotiation.cc:299] Negotiation successful
0809 09:32:58.560847 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260809 09:32:58.561947 32071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:58.537812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.189:44285 (local address 127.20.125.131:43511)
0809 09:32:58.539322 (+ 1510us) negotiation.cc:107] Waiting for socket to connect
0809 09:32:58.539338 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 09:32:58.541471 (+ 2133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:32:58.542735 (+ 1264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:32:58.542738 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:32:58.542932 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:32:58.543052 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:32:58.543058 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:32:58.545120 (+ 2062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:32:58.545124 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:32:58.546043 (+ 919us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:32:58.546055 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:32:58.548521 (+ 2466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:32:58.548524 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:32:58.548611 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:58.549125 (+ 514us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 09:32:58.549136 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:32:58.555492 (+ 6356us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 09:32:58.558805 (+ 3313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:32:58.558809 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:32:58.558811 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:32:58.559271 (+ 460us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:32:58.559724 (+ 453us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:32:58.559731 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:32:58.559733 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:32:58.559842 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:32:58.560374 (+ 532us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 09:32:58.560380 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 09:32:58.560484 (+ 104us) client_negotiation.cc:769] Sending connection context
0809 09:32:58.561197 (+ 713us) client_negotiation.cc:240] Negotiation successful
0809 09:32:58.561366 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1415,"mutex_wait_us":2,"thread_start_us":139,"threads_started":1}
I20260809 09:32:58.562084 30241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:58.536927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.131:50339 (local address 127.20.125.190:37175)
0809 09:32:58.537092 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0809 09:32:58.537097 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 09:32:58.538648 (+ 1551us) server_negotiation.cc:407] Connection header received
0809 09:32:58.541656 (+ 3008us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:32:58.541659 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:32:58.541708 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:32:58.541758 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:32:58.543285 (+ 1527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:58.544048 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:58.547266 (+ 3218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:58.547611 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:58.547702 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:58.557945 (+ 10243us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:32:58.557951 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:32:58.557954 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:32:58.557988 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:32:58.559990 (+ 2002us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:32:58.560545 (+ 555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:32:58.560551 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:32:58.560556 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:32:58.560659 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:32:58.560997 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:32:58.561001 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:32:58.561002 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:32:58.561254 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:32:58.561349 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0809 09:32:58.561741 (+ 392us) server_negotiation.cc:299] Negotiation successful
0809 09:32:58.561895 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260809 09:32:58.562812 32080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:58.536639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:37175 (local address 127.20.125.131:50339)
0809 09:32:58.538293 (+ 1654us) negotiation.cc:107] Waiting for socket to connect
0809 09:32:58.538358 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0809 09:32:58.541552 (+ 3194us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:32:58.542137 (+ 585us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:32:58.542140 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:32:58.542347 (+ 207us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:32:58.543164 (+ 817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:32:58.543171 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:32:58.545077 (+ 1906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:32:58.545083 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:32:58.547134 (+ 2051us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:32:58.547145 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:32:58.548358 (+ 1213us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:32:58.548361 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:32:58.548901 (+ 540us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:58.550219 (+ 1318us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 09:32:58.550235 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:32:58.556349 (+ 6114us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 09:32:58.560130 (+ 3781us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:32:58.560134 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:32:58.560136 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:32:58.560421 (+ 285us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:32:58.560783 (+ 362us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:32:58.560786 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:32:58.560789 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:32:58.560883 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:32:58.561411 (+ 528us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 09:32:58.561414 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 09:32:58.561511 (+ 97us) client_negotiation.cc:769] Sending connection context
0809 09:32:58.562390 (+ 879us) client_negotiation.cc:240] Negotiation successful
0809 09:32:58.562590 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1402,"mutex_wait_us":1017,"thread_start_us":919,"threads_started":1}
I20260809 09:32:58.563099 32068 heartbeater.cc:347] Connected to a master server at 127.20.125.189:44285
W20260809 09:32:58.563215 30181 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 09:32:58.563371 32068 heartbeater.cc:464] Registering TS with master...
I20260809 09:32:58.563556 32064 heartbeater.cc:347] Connected to a master server at 127.20.125.190:37175
I20260809 09:32:58.563668 32064 heartbeater.cc:464] Registering TS with master...
I20260809 09:32:58.563099 32063 heartbeater.cc:347] Connected to a master server at 127.20.125.188:36019
I20260809 09:32:58.563989 32063 heartbeater.cc:464] Registering TS with master...
I20260809 09:32:58.564042 32064 heartbeater.cc:511] Master 127.20.125.190:37175 requested a full tablet report, sending...
I20260809 09:32:58.564041 32068 heartbeater.cc:511] Master 127.20.125.189:44285 requested a full tablet report, sending...
I20260809 09:32:58.564155 32063 heartbeater.cc:511] Master 127.20.125.188:36019 requested a full tablet report, sending...
I20260809 09:32:58.565220 30182 ts_manager.cc:194] Re-registered known tserver with Master: 825ed73d552f4f8cbce972a3dc70ccd8 (127.20.125.131:44279)
I20260809 09:32:58.565407 31389 ts_manager.cc:194] Registered new tserver with Master: 825ed73d552f4f8cbce972a3dc70ccd8 (127.20.125.131:44279)
I20260809 09:32:58.566027 30182 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.20.125.131@KRBTEST.COM'} at 127.20.125.131:50339
I20260809 09:32:58.566063 31041 sys_catalog.cc:455] T 00000000000000000000000000000000 P ec1960de545a42ba9ac3e3e9adc99056 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ec1960de545a42ba9ac3e3e9adc99056" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "ec1960de545a42ba9ac3e3e9adc99056" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 37175 } } peers { permanent_uuid: "3c029f42802647dea9a4c2f436be31eb" member_type: VOTER last_known_addr { host: "127.20.125.189" port: 44285 } attrs { promote: false } } peers { permanent_uuid: "dece4cc043764f62a3667954979c5562" member_type: NON_VOTER last_known_addr { host: "127.20.125.188" port: 36019 } attrs { promote: true } } }
I20260809 09:32:58.566175 31041 sys_catalog.cc:458] T 00000000000000000000000000000000 P ec1960de545a42ba9ac3e3e9adc99056 [sys.catalog]: This master's current role is: LEADER
I20260809 09:32:58.566663 31112 ts_manager.cc:194] Registered new tserver with Master: 825ed73d552f4f8cbce972a3dc70ccd8 (127.20.125.131:44279)
I20260809 09:32:58.569730 31041 raft_consensus.cc:1050] T 00000000000000000000000000000000 P ec1960de545a42ba9ac3e3e9adc99056: attempting to promote NON_VOTER dece4cc043764f62a3667954979c5562 to VOTER
I20260809 09:32:58.570109 31041 consensus_queue.cc:237] T 00000000000000000000000000000000 P ec1960de545a42ba9ac3e3e9adc99056 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "ec1960de545a42ba9ac3e3e9adc99056" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 37175 } } peers { permanent_uuid: "3c029f42802647dea9a4c2f436be31eb" member_type: VOTER last_known_addr { host: "127.20.125.189" port: 44285 } attrs { promote: false } } peers { permanent_uuid: "dece4cc043764f62a3667954979c5562" member_type: VOTER last_known_addr { host: "127.20.125.188" port: 36019 } attrs { promote: false } }
I20260809 09:32:58.571584 31117 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 3c029f42802647dea9a4c2f436be31eb [term 1 FOLLOWER]: Refusing update from remote peer ec1960de545a42ba9ac3e3e9adc99056: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260809 09:32:58.571695 31400 raft_consensus.cc:1261] T 00000000000000000000000000000000 P dece4cc043764f62a3667954979c5562 [term 1 LEARNER]: Refusing update from remote peer ec1960de545a42ba9ac3e3e9adc99056: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260809 09:32:58.572074 32096 consensus_queue.cc:1035] T 00000000000000000000000000000000 P ec1960de545a42ba9ac3e3e9adc99056 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c029f42802647dea9a4c2f436be31eb" member_type: VOTER last_known_addr { host: "127.20.125.189" port: 44285 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260809 09:32:58.572243 32097 consensus_queue.cc:1035] T 00000000000000000000000000000000 P ec1960de545a42ba9ac3e3e9adc99056 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dece4cc043764f62a3667954979c5562" member_type: VOTER last_known_addr { host: "127.20.125.188" port: 36019 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260809 09:32:58.574604 32097 raft_consensus.cc:2932] T 00000000000000000000000000000000 P ec1960de545a42ba9ac3e3e9adc99056 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 7 to 9, dece4cc043764f62a3667954979c5562 (127.20.125.188) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "ec1960de545a42ba9ac3e3e9adc99056" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 37175 } } peers { permanent_uuid: "3c029f42802647dea9a4c2f436be31eb" member_type: VOTER last_known_addr { host: "127.20.125.189" port: 44285 } attrs { promote: false } } peers { permanent_uuid: "dece4cc043764f62a3667954979c5562" member_type: VOTER last_known_addr { host: "127.20.125.188" port: 36019 } attrs { promote: false } } }
I20260809 09:32:58.574865 31400 raft_consensus.cc:2932] T 00000000000000000000000000000000 P dece4cc043764f62a3667954979c5562 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, dece4cc043764f62a3667954979c5562 (127.20.125.188) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "ec1960de545a42ba9ac3e3e9adc99056" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 37175 } } peers { permanent_uuid: "3c029f42802647dea9a4c2f436be31eb" member_type: VOTER last_known_addr { host: "127.20.125.189" port: 44285 } attrs { promote: false } } peers { permanent_uuid: "dece4cc043764f62a3667954979c5562" member_type: VOTER last_known_addr { host: "127.20.125.188" port: 36019 } attrs { promote: false } } }
I20260809 09:32:58.576350 32097 sys_catalog.cc:455] T 00000000000000000000000000000000 P ec1960de545a42ba9ac3e3e9adc99056 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "ec1960de545a42ba9ac3e3e9adc99056" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "ec1960de545a42ba9ac3e3e9adc99056" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 37175 } } peers { permanent_uuid: "3c029f42802647dea9a4c2f436be31eb" member_type: VOTER last_known_addr { host: "127.20.125.189" port: 44285 } attrs { promote: false } } peers { permanent_uuid: "dece4cc043764f62a3667954979c5562" member_type: VOTER last_known_addr { host: "127.20.125.188" port: 36019 } attrs { promote: false } } }
I20260809 09:32:58.576488 32097 sys_catalog.cc:458] T 00000000000000000000000000000000 P ec1960de545a42ba9ac3e3e9adc99056 [sys.catalog]: This master's current role is: LEADER
I20260809 09:32:58.576771 32097 sys_catalog.cc:455] T 00000000000000000000000000000000 P ec1960de545a42ba9ac3e3e9adc99056 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ec1960de545a42ba9ac3e3e9adc99056" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "ec1960de545a42ba9ac3e3e9adc99056" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 37175 } } peers { permanent_uuid: "3c029f42802647dea9a4c2f436be31eb" member_type: VOTER last_known_addr { host: "127.20.125.189" port: 44285 } attrs { promote: false } } peers { permanent_uuid: "dece4cc043764f62a3667954979c5562" member_type: VOTER last_known_addr { host: "127.20.125.188" port: 36019 } attrs { promote: false } } }
I20260809 09:32:58.576844 31442 sys_catalog.cc:455] T 00000000000000000000000000000000 P dece4cc043764f62a3667954979c5562 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ec1960de545a42ba9ac3e3e9adc99056" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "ec1960de545a42ba9ac3e3e9adc99056" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 37175 } } peers { permanent_uuid: "3c029f42802647dea9a4c2f436be31eb" member_type: VOTER last_known_addr { host: "127.20.125.189" port: 44285 } attrs { promote: false } } peers { permanent_uuid: "dece4cc043764f62a3667954979c5562" member_type: VOTER last_known_addr { host: "127.20.125.188" port: 36019 } attrs { promote: false } } }
I20260809 09:32:58.576912 32097 sys_catalog.cc:458] T 00000000000000000000000000000000 P ec1960de545a42ba9ac3e3e9adc99056 [sys.catalog]: This master's current role is: LEADER
I20260809 09:32:58.576965 31442 sys_catalog.cc:458] T 00000000000000000000000000000000 P dece4cc043764f62a3667954979c5562 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 09:32:58.577286 31041 sys_catalog.cc:455] T 00000000000000000000000000000000 P ec1960de545a42ba9ac3e3e9adc99056 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ec1960de545a42ba9ac3e3e9adc99056" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "ec1960de545a42ba9ac3e3e9adc99056" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 37175 } } peers { permanent_uuid: "3c029f42802647dea9a4c2f436be31eb" member_type: VOTER last_known_addr { host: "127.20.125.189" port: 44285 } attrs { promote: false } } peers { permanent_uuid: "dece4cc043764f62a3667954979c5562" member_type: VOTER last_known_addr { host: "127.20.125.188" port: 36019 } attrs { promote: false } } }
I20260809 09:32:58.577417 31041 sys_catalog.cc:458] T 00000000000000000000000000000000 P ec1960de545a42ba9ac3e3e9adc99056 [sys.catalog]: This master's current role is: LEADER
I20260809 09:32:58.577860 31117 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 3c029f42802647dea9a4c2f436be31eb [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, dece4cc043764f62a3667954979c5562 (127.20.125.188) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "ec1960de545a42ba9ac3e3e9adc99056" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 37175 } } peers { permanent_uuid: "3c029f42802647dea9a4c2f436be31eb" member_type: VOTER last_known_addr { host: "127.20.125.189" port: 44285 } attrs { promote: false } } peers { permanent_uuid: "dece4cc043764f62a3667954979c5562" member_type: VOTER last_known_addr { host: "127.20.125.188" port: 36019 } attrs { promote: false } } }
I20260809 09:32:58.579685 32112 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3c029f42802647dea9a4c2f436be31eb [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ec1960de545a42ba9ac3e3e9adc99056" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "ec1960de545a42ba9ac3e3e9adc99056" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 37175 } } peers { permanent_uuid: "3c029f42802647dea9a4c2f436be31eb" member_type: VOTER last_known_addr { host: "127.20.125.189" port: 44285 } attrs { promote: false } } peers { permanent_uuid: "dece4cc043764f62a3667954979c5562" member_type: VOTER last_known_addr { host: "127.20.125.188" port: 36019 } attrs { promote: false } } }
I20260809 09:32:58.579834 32112 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3c029f42802647dea9a4c2f436be31eb [sys.catalog]: This master's current role is: FOLLOWER
I20260809 09:32:58.591069 31032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:58.585137 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.131:44279 (local address 127.0.0.1:39254)
0809 09:32:58.585281 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0809 09:32:58.585298 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 09:32:58.585406 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:32:58.586558 (+ 1152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:32:58.586561 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:32:58.586579 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 09:32:58.586693 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:32:58.586699 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:32:58.588171 (+ 1472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:32:58.588175 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:32:58.589426 (+ 1251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:32:58.589437 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:32:58.590663 (+ 1226us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:32:58.590668 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:32:58.590769 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:58.590799 (+ 30us) client_negotiation.cc:769] Sending connection context
0809 09:32:58.590854 (+ 55us) client_negotiation.cc:240] Negotiation successful
0809 09:32:58.590919 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51}
I20260809 09:32:58.592470 32116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:58.585707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.130:44297 (local address 127.0.0.1:53070)
0809 09:32:58.586164 (+ 457us) negotiation.cc:107] Waiting for socket to connect
0809 09:32:58.586180 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 09:32:58.586406 (+ 226us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:32:58.587357 (+ 951us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:32:58.587362 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:32:58.587381 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 09:32:58.587486 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:32:58.587492 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:32:58.589317 (+ 1825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:32:58.589321 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:32:58.590824 (+ 1503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:32:58.590837 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:32:58.592067 (+ 1230us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:32:58.592071 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:32:58.592173 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:58.592200 (+ 27us) client_negotiation.cc:769] Sending connection context
0809 09:32:58.592251 (+ 51us) client_negotiation.cc:240] Negotiation successful
0809 09:32:58.592321 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":370,"thread_start_us":281,"threads_started":1}
I20260809 09:32:58.592712 32115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:58.585339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39254 (local address 127.20.125.131:44279)
0809 09:32:58.585787 (+ 448us) server_negotiation.cc:206] Beginning negotiation
0809 09:32:58.585802 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 09:32:58.585828 (+ 26us) server_negotiation.cc:407] Connection header received
0809 09:32:58.585916 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:32:58.585921 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:32:58.586169 (+ 248us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:32:58.586616 (+ 447us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 09:32:58.586786 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:58.588077 (+ 1291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:58.589557 (+ 1480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:58.590489 (+ 932us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:58.591407 (+ 918us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:58.591553 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0809 09:32:58.591606 (+ 53us) server_negotiation.cc:299] Negotiation successful
0809 09:32:58.591716 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":329,"thread_start_us":99,"threads_started":1}
I20260809 09:32:58.593139 32117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:58.586338 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53070 (local address 127.20.125.130:44297)
0809 09:32:58.586676 (+ 338us) server_negotiation.cc:206] Beginning negotiation
0809 09:32:58.586701 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0809 09:32:58.586727 (+ 26us) server_negotiation.cc:407] Connection header received
0809 09:32:58.586811 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:32:58.586816 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:32:58.587025 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:32:58.587186 (+ 161us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 09:32:58.587670 (+ 484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:58.589158 (+ 1488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:58.590998 (+ 1840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:58.591910 (+ 912us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:58.592717 (+ 807us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:58.592848 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0809 09:32:58.592898 (+ 50us) server_negotiation.cc:299] Negotiation successful
0809 09:32:58.592988 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":236,"thread_start_us":146,"threads_started":1}
I20260809 09:32:58.595218 32118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:58.586848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.129:42545 (local address 127.0.0.1:38874)
0809 09:32:58.587112 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0809 09:32:58.587137 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0809 09:32:58.587224 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:32:58.588092 (+ 868us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:32:58.588095 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:32:58.589585 (+ 1490us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 09:32:58.589690 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:32:58.589697 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:32:58.591845 (+ 2148us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:32:58.591848 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:32:58.593557 (+ 1709us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:32:58.593572 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:32:58.594799 (+ 1227us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:32:58.594804 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:32:58.594905 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:58.594932 (+ 27us) client_negotiation.cc:769] Sending connection context
0809 09:32:58.594987 (+ 55us) client_negotiation.cc:240] Negotiation successful
0809 09:32:58.595062 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":104,"threads_started":1}
I20260809 09:32:58.596179 32120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:58.587324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38874 (local address 127.20.125.129:42545)
0809 09:32:58.587632 (+ 308us) server_negotiation.cc:206] Beginning negotiation
0809 09:32:58.587650 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0809 09:32:58.587673 (+ 23us) server_negotiation.cc:407] Connection header received
0809 09:32:58.587748 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:32:58.587753 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:32:58.587931 (+ 178us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:32:58.589858 (+ 1927us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 09:32:58.590005 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:58.591698 (+ 1693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:58.593739 (+ 2041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:58.594642 (+ 903us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:58.595702 (+ 1060us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:58.595844 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0809 09:32:58.595901 (+ 57us) server_negotiation.cc:299] Negotiation successful
0809 09:32:58.596010 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":192,"thread_start_us":78,"threads_started":1}
I20260809 09:32:58.596143 31960 tablet_service.cc:1467] Processing CreateTablet for tablet ca167cef7420445b844d66e55a512774 (DEFAULT_TABLE table=test-table [id=ceb170d10b0442cb86ef19577f546dd9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 09:32:58.596227 31765 tablet_service.cc:1467] Processing CreateTablet for tablet ca167cef7420445b844d66e55a512774 (DEFAULT_TABLE table=test-table [id=ceb170d10b0442cb86ef19577f546dd9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 09:32:58.597430 31765 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ca167cef7420445b844d66e55a512774. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 09:32:58.597434 31960 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ca167cef7420445b844d66e55a512774. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 09:32:58.599347 31552 tablet_service.cc:1467] Processing CreateTablet for tablet ca167cef7420445b844d66e55a512774 (DEFAULT_TABLE table=test-table [id=ceb170d10b0442cb86ef19577f546dd9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 09:32:58.600478 31552 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ca167cef7420445b844d66e55a512774. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 09:32:58.605204 32123 tablet_bootstrap.cc:492] T ca167cef7420445b844d66e55a512774 P 825ed73d552f4f8cbce972a3dc70ccd8: Bootstrap starting.
I20260809 09:32:58.607694 32123 tablet_bootstrap.cc:654] T ca167cef7420445b844d66e55a512774 P 825ed73d552f4f8cbce972a3dc70ccd8: Neither blocks nor log segments found. Creating new log.
I20260809 09:32:58.608520 32125 tablet_bootstrap.cc:492] T ca167cef7420445b844d66e55a512774 P 8bbe397caa0242579d9743094319a762: Bootstrap starting.
I20260809 09:32:58.608781 32123 log.cc:824] T ca167cef7420445b844d66e55a512774 P 825ed73d552f4f8cbce972a3dc70ccd8: Log is configured to *not* fsync() on all Append() calls
I20260809 09:32:58.609694 32126 tablet_bootstrap.cc:492] T ca167cef7420445b844d66e55a512774 P 143f334e51d54d17911a2d2ebb0b6554: Bootstrap starting.
I20260809 09:32:58.611214 32123 tablet_bootstrap.cc:492] T ca167cef7420445b844d66e55a512774 P 825ed73d552f4f8cbce972a3dc70ccd8: No bootstrap required, opened a new log
I20260809 09:32:58.611449 32123 ts_tablet_manager.cc:1397] T ca167cef7420445b844d66e55a512774 P 825ed73d552f4f8cbce972a3dc70ccd8: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260809 09:32:58.611472 32125 tablet_bootstrap.cc:654] T ca167cef7420445b844d66e55a512774 P 8bbe397caa0242579d9743094319a762: Neither blocks nor log segments found. Creating new log.
I20260809 09:32:58.611845 32126 tablet_bootstrap.cc:654] T ca167cef7420445b844d66e55a512774 P 143f334e51d54d17911a2d2ebb0b6554: Neither blocks nor log segments found. Creating new log.
I20260809 09:32:58.612401 32125 log.cc:824] T ca167cef7420445b844d66e55a512774 P 8bbe397caa0242579d9743094319a762: Log is configured to *not* fsync() on all Append() calls
I20260809 09:32:58.612695 32126 log.cc:824] T ca167cef7420445b844d66e55a512774 P 143f334e51d54d17911a2d2ebb0b6554: Log is configured to *not* fsync() on all Append() calls
I20260809 09:32:58.615023 32126 tablet_bootstrap.cc:492] T ca167cef7420445b844d66e55a512774 P 143f334e51d54d17911a2d2ebb0b6554: No bootstrap required, opened a new log
I20260809 09:32:58.615260 32126 ts_tablet_manager.cc:1397] T ca167cef7420445b844d66e55a512774 P 143f334e51d54d17911a2d2ebb0b6554: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260809 09:32:58.616390 32125 tablet_bootstrap.cc:492] T ca167cef7420445b844d66e55a512774 P 8bbe397caa0242579d9743094319a762: No bootstrap required, opened a new log
I20260809 09:32:58.616299 32123 raft_consensus.cc:348] T ca167cef7420445b844d66e55a512774 P 825ed73d552f4f8cbce972a3dc70ccd8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "825ed73d552f4f8cbce972a3dc70ccd8" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 44279 } } peers { permanent_uuid: "143f334e51d54d17911a2d2ebb0b6554" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 42545 } } peers { permanent_uuid: "8bbe397caa0242579d9743094319a762" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 44297 } }
I20260809 09:32:58.616585 32123 raft_consensus.cc:374] T ca167cef7420445b844d66e55a512774 P 825ed73d552f4f8cbce972a3dc70ccd8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 09:32:58.616585 32125 ts_tablet_manager.cc:1397] T ca167cef7420445b844d66e55a512774 P 8bbe397caa0242579d9743094319a762: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260809 09:32:58.616654 32123 raft_consensus.cc:729] T ca167cef7420445b844d66e55a512774 P 825ed73d552f4f8cbce972a3dc70ccd8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 825ed73d552f4f8cbce972a3dc70ccd8, State: Initialized, Role: FOLLOWER
I20260809 09:32:58.617151 32123 consensus_queue.cc:260] T ca167cef7420445b844d66e55a512774 P 825ed73d552f4f8cbce972a3dc70ccd8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "825ed73d552f4f8cbce972a3dc70ccd8" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 44279 } } peers { permanent_uuid: "143f334e51d54d17911a2d2ebb0b6554" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 42545 } } peers { permanent_uuid: "8bbe397caa0242579d9743094319a762" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 44297 } }
I20260809 09:32:58.618299 32123 ts_tablet_manager.cc:1428] T ca167cef7420445b844d66e55a512774 P 825ed73d552f4f8cbce972a3dc70ccd8: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260809 09:32:58.618525 32064 heartbeater.cc:503] Master 127.20.125.190:37175 was elected leader, sending a full tablet report...
I20260809 09:32:58.620532 32126 raft_consensus.cc:348] T ca167cef7420445b844d66e55a512774 P 143f334e51d54d17911a2d2ebb0b6554 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "825ed73d552f4f8cbce972a3dc70ccd8" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 44279 } } peers { permanent_uuid: "143f334e51d54d17911a2d2ebb0b6554" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 42545 } } peers { permanent_uuid: "8bbe397caa0242579d9743094319a762" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 44297 } }
I20260809 09:32:58.620847 32126 raft_consensus.cc:374] T ca167cef7420445b844d66e55a512774 P 143f334e51d54d17911a2d2ebb0b6554 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 09:32:58.620916 32126 raft_consensus.cc:729] T ca167cef7420445b844d66e55a512774 P 143f334e51d54d17911a2d2ebb0b6554 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 143f334e51d54d17911a2d2ebb0b6554, State: Initialized, Role: FOLLOWER
I20260809 09:32:58.621511 32126 consensus_queue.cc:260] T ca167cef7420445b844d66e55a512774 P 143f334e51d54d17911a2d2ebb0b6554 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "825ed73d552f4f8cbce972a3dc70ccd8" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 44279 } } peers { permanent_uuid: "143f334e51d54d17911a2d2ebb0b6554" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 42545 } } peers { permanent_uuid: "8bbe397caa0242579d9743094319a762" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 44297 } }
I20260809 09:32:58.622227 32125 raft_consensus.cc:348] T ca167cef7420445b844d66e55a512774 P 8bbe397caa0242579d9743094319a762 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "825ed73d552f4f8cbce972a3dc70ccd8" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 44279 } } peers { permanent_uuid: "143f334e51d54d17911a2d2ebb0b6554" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 42545 } } peers { permanent_uuid: "8bbe397caa0242579d9743094319a762" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 44297 } }
I20260809 09:32:58.622516 32125 raft_consensus.cc:374] T ca167cef7420445b844d66e55a512774 P 8bbe397caa0242579d9743094319a762 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 09:32:58.622586 32125 raft_consensus.cc:729] T ca167cef7420445b844d66e55a512774 P 8bbe397caa0242579d9743094319a762 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8bbe397caa0242579d9743094319a762, State: Initialized, Role: FOLLOWER
I20260809 09:32:58.623179 32126 ts_tablet_manager.cc:1428] T ca167cef7420445b844d66e55a512774 P 143f334e51d54d17911a2d2ebb0b6554: Time spent starting tablet: real 0.008s user 0.007s sys 0.000s
I20260809 09:32:58.623346 31658 heartbeater.cc:503] Master 127.20.125.190:37175 was elected leader, sending a full tablet report...
I20260809 09:32:58.623829 32125 consensus_queue.cc:260] T ca167cef7420445b844d66e55a512774 P 8bbe397caa0242579d9743094319a762 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "825ed73d552f4f8cbce972a3dc70ccd8" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 44279 } } peers { permanent_uuid: "143f334e51d54d17911a2d2ebb0b6554" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 42545 } } peers { permanent_uuid: "8bbe397caa0242579d9743094319a762" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 44297 } }
I20260809 09:32:58.624892 32125 ts_tablet_manager.cc:1428] T ca167cef7420445b844d66e55a512774 P 8bbe397caa0242579d9743094319a762: Time spent starting tablet: real 0.008s user 0.003s sys 0.004s
I20260809 09:32:58.625306 31863 heartbeater.cc:503] Master 127.20.125.190:37175 was elected leader, sending a full tablet report...
I20260809 09:32:58.656132 32140 raft_consensus.cc:482] T ca167cef7420445b844d66e55a512774 P 143f334e51d54d17911a2d2ebb0b6554 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 09:32:58.656337 32140 raft_consensus.cc:504] T ca167cef7420445b844d66e55a512774 P 143f334e51d54d17911a2d2ebb0b6554 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "825ed73d552f4f8cbce972a3dc70ccd8" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 44279 } } peers { permanent_uuid: "143f334e51d54d17911a2d2ebb0b6554" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 42545 } } peers { permanent_uuid: "8bbe397caa0242579d9743094319a762" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 44297 } }
I20260809 09:32:58.657617 32140 leader_election.cc:288] T ca167cef7420445b844d66e55a512774 P 143f334e51d54d17911a2d2ebb0b6554 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 825ed73d552f4f8cbce972a3dc70ccd8 (127.20.125.131:44279), 8bbe397caa0242579d9743094319a762 (127.20.125.130:44297)
I20260809 09:32:58.662107 32142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:58.657868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.131:44279 (local address 127.20.125.129:42263)
0809 09:32:58.658308 (+ 440us) negotiation.cc:107] Waiting for socket to connect
0809 09:32:58.658320 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 09:32:58.658419 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:32:58.658762 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:32:58.658768 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:32:58.658790 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 09:32:58.658938 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:32:58.658944 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:32:58.659988 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:32:58.659991 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:32:58.660830 (+ 839us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:32:58.660839 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:32:58.661624 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:32:58.661631 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:32:58.661738 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:58.661762 (+ 24us) client_negotiation.cc:769] Sending connection context
0809 09:32:58.661845 (+ 83us) client_negotiation.cc:240] Negotiation successful
0809 09:32:58.661902 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":373,"mutex_wait_us":26,"thread_start_us":153,"threads_started":1}
I20260809 09:32:58.662243 32117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:58.658039 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.129:58737 (local address 127.20.125.130:44297)
0809 09:32:58.658200 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0809 09:32:58.658206 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 09:32:58.658489 (+ 283us) server_negotiation.cc:407] Connection header received
0809 09:32:58.658664 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:32:58.658666 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:32:58.658739 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:32:58.658819 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 09:32:58.659174 (+ 355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:58.659697 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:58.661081 (+ 1384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:58.661528 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:58.661722 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:58.661773 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0809 09:32:58.662026 (+ 253us) server_negotiation.cc:299] Negotiation successful
0809 09:32:58.662114 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260809 09:32:58.662302 32144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:58.657955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.130:44297 (local address 127.20.125.129:58737)
0809 09:32:58.658349 (+ 394us) negotiation.cc:107] Waiting for socket to connect
0809 09:32:58.658361 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 09:32:58.658550 (+ 189us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:32:58.658855 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:32:58.658858 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:32:58.658883 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 09:32:58.659004 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:32:58.659014 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:32:58.659816 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:32:58.659820 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:32:58.660825 (+ 1005us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:32:58.660835 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:32:58.661730 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:32:58.661734 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:32:58.661868 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:58.661900 (+ 32us) client_negotiation.cc:769] Sending connection context
0809 09:32:58.661984 (+ 84us) client_negotiation.cc:240] Negotiation successful
0809 09:32:58.662057 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":340,"mutex_wait_us":4,"thread_start_us":135,"threads_started":1}
I20260809 09:32:58.662122 32115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:58.658023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.129:42263 (local address 127.20.125.131:44279)
0809 09:32:58.658203 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0809 09:32:58.658210 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 09:32:58.658413 (+ 203us) server_negotiation.cc:407] Connection header received
0809 09:32:58.658570 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:32:58.658576 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:32:58.658642 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:32:58.658721 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 09:32:58.659074 (+ 353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:58.659847 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:58.661019 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:58.661490 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:58.661598 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:58.661685 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0809 09:32:58.661884 (+ 199us) server_negotiation.cc:299] Negotiation successful
0809 09:32:58.661983 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101}
I20260809 09:32:58.663075 31793 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ca167cef7420445b844d66e55a512774" candidate_uuid: "143f334e51d54d17911a2d2ebb0b6554" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8bbe397caa0242579d9743094319a762" is_pre_election: true
I20260809 09:32:58.663156 31994 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ca167cef7420445b844d66e55a512774" candidate_uuid: "143f334e51d54d17911a2d2ebb0b6554" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "825ed73d552f4f8cbce972a3dc70ccd8" is_pre_election: true
I20260809 09:32:58.663416 31793 raft_consensus.cc:2454] T ca167cef7420445b844d66e55a512774 P 8bbe397caa0242579d9743094319a762 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 143f334e51d54d17911a2d2ebb0b6554 in term 0.
I20260809 09:32:58.663532 31994 raft_consensus.cc:2454] T ca167cef7420445b844d66e55a512774 P 825ed73d552f4f8cbce972a3dc70ccd8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 143f334e51d54d17911a2d2ebb0b6554 in term 0.
I20260809 09:32:58.663923 31493 leader_election.cc:302] T ca167cef7420445b844d66e55a512774 P 143f334e51d54d17911a2d2ebb0b6554 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 143f334e51d54d17911a2d2ebb0b6554, 8bbe397caa0242579d9743094319a762; no voters:
I20260809 09:32:58.664187 32140 raft_consensus.cc:2781] T ca167cef7420445b844d66e55a512774 P 143f334e51d54d17911a2d2ebb0b6554 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 09:32:58.664269 32140 raft_consensus.cc:482] T ca167cef7420445b844d66e55a512774 P 143f334e51d54d17911a2d2ebb0b6554 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 09:32:58.664311 32140 raft_consensus.cc:3037] T ca167cef7420445b844d66e55a512774 P 143f334e51d54d17911a2d2ebb0b6554 [term 0 FOLLOWER]: Advancing to term 1
I20260809 09:32:58.665439 32140 raft_consensus.cc:504] T ca167cef7420445b844d66e55a512774 P 143f334e51d54d17911a2d2ebb0b6554 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "825ed73d552f4f8cbce972a3dc70ccd8" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 44279 } } peers { permanent_uuid: "143f334e51d54d17911a2d2ebb0b6554" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 42545 } } peers { permanent_uuid: "8bbe397caa0242579d9743094319a762" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 44297 } }
I20260809 09:32:58.665839 32140 leader_election.cc:288] T ca167cef7420445b844d66e55a512774 P 143f334e51d54d17911a2d2ebb0b6554 [CANDIDATE]: Term 1 election: Requested vote from peers 825ed73d552f4f8cbce972a3dc70ccd8 (127.20.125.131:44279), 8bbe397caa0242579d9743094319a762 (127.20.125.130:44297)
I20260809 09:32:58.666206 31994 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ca167cef7420445b844d66e55a512774" candidate_uuid: "143f334e51d54d17911a2d2ebb0b6554" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "825ed73d552f4f8cbce972a3dc70ccd8"
I20260809 09:32:58.666357 31994 raft_consensus.cc:3037] T ca167cef7420445b844d66e55a512774 P 825ed73d552f4f8cbce972a3dc70ccd8 [term 0 FOLLOWER]: Advancing to term 1
I20260809 09:32:58.666322 31793 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ca167cef7420445b844d66e55a512774" candidate_uuid: "143f334e51d54d17911a2d2ebb0b6554" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8bbe397caa0242579d9743094319a762"
I20260809 09:32:58.666529 31793 raft_consensus.cc:3037] T ca167cef7420445b844d66e55a512774 P 8bbe397caa0242579d9743094319a762 [term 0 FOLLOWER]: Advancing to term 1
I20260809 09:32:58.667595 31994 raft_consensus.cc:2454] T ca167cef7420445b844d66e55a512774 P 825ed73d552f4f8cbce972a3dc70ccd8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 143f334e51d54d17911a2d2ebb0b6554 in term 1.
I20260809 09:32:58.667904 31491 leader_election.cc:302] T ca167cef7420445b844d66e55a512774 P 143f334e51d54d17911a2d2ebb0b6554 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 143f334e51d54d17911a2d2ebb0b6554, 825ed73d552f4f8cbce972a3dc70ccd8; no voters:
I20260809 09:32:58.667970 31793 raft_consensus.cc:2454] T ca167cef7420445b844d66e55a512774 P 8bbe397caa0242579d9743094319a762 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 143f334e51d54d17911a2d2ebb0b6554 in term 1.
I20260809 09:32:58.668118 32140 raft_consensus.cc:2781] T ca167cef7420445b844d66e55a512774 P 143f334e51d54d17911a2d2ebb0b6554 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 09:32:58.668358 32140 raft_consensus.cc:686] T ca167cef7420445b844d66e55a512774 P 143f334e51d54d17911a2d2ebb0b6554 [term 1 LEADER]: Becoming Leader. State: Replica: 143f334e51d54d17911a2d2ebb0b6554, State: Running, Role: LEADER
I20260809 09:32:58.668833 32140 consensus_queue.cc:237] T ca167cef7420445b844d66e55a512774 P 143f334e51d54d17911a2d2ebb0b6554 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "825ed73d552f4f8cbce972a3dc70ccd8" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 44279 } } peers { permanent_uuid: "143f334e51d54d17911a2d2ebb0b6554" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 42545 } } peers { permanent_uuid: "8bbe397caa0242579d9743094319a762" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 44297 } }
I20260809 09:32:58.672669 30182 catalog_manager.cc:5495] T ca167cef7420445b844d66e55a512774 P 143f334e51d54d17911a2d2ebb0b6554 reported cstate change: term changed from 0 to 1, leader changed from <none> to 143f334e51d54d17911a2d2ebb0b6554 (127.20.125.129). New cstate: current_term: 1 leader_uuid: "143f334e51d54d17911a2d2ebb0b6554" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "825ed73d552f4f8cbce972a3dc70ccd8" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 44279 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "143f334e51d54d17911a2d2ebb0b6554" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 42545 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8bbe397caa0242579d9743094319a762" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 44297 } health_report { overall_health: UNKNOWN } } }
I20260809 09:32:58.695993 32120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:58.692365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38880 (local address 127.20.125.129:42545)
0809 09:32:58.692492 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0809 09:32:58.692496 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 09:32:58.692508 (+ 12us) server_negotiation.cc:407] Connection header received
0809 09:32:58.692580 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:32:58.692582 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:32:58.692638 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:32:58.692723 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 09:32:58.692953 (+ 230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:58.693467 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:58.694359 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:58.694674 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:58.694799 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:58.695137 (+ 338us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 09:32:58.695563 (+ 426us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 09:32:58.695698 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0809 09:32:58.695771 (+ 73us) server_negotiation.cc:299] Negotiation successful
0809 09:32:58.695833 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260809 09:32:58.702862 31793 raft_consensus.cc:1261] T ca167cef7420445b844d66e55a512774 P 8bbe397caa0242579d9743094319a762 [term 1 FOLLOWER]: Refusing update from remote peer 143f334e51d54d17911a2d2ebb0b6554: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 09:32:58.702872 31994 raft_consensus.cc:1261] T ca167cef7420445b844d66e55a512774 P 825ed73d552f4f8cbce972a3dc70ccd8 [term 1 FOLLOWER]: Refusing update from remote peer 143f334e51d54d17911a2d2ebb0b6554: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 09:32:58.703688 32146 consensus_queue.cc:1035] T ca167cef7420445b844d66e55a512774 P 143f334e51d54d17911a2d2ebb0b6554 [LEADER]: Connected to new peer: Peer: permanent_uuid: "825ed73d552f4f8cbce972a3dc70ccd8" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 44279 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 09:32:58.703987 32140 consensus_queue.cc:1035] T ca167cef7420445b844d66e55a512774 P 143f334e51d54d17911a2d2ebb0b6554 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8bbe397caa0242579d9743094319a762" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 44297 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 09:32:58.718353 32153 mvcc.cc:205] Tried to move back new op lower bound from 7316553640757690368 to 7316553640629428224. Current Snapshot: MvccSnapshot[applied={T|T < 7316553640757690368}]
I20260809 09:32:58.725637 30180 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52710:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 09:32:58.725880 30180 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52710:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 09:32:58.731256 30180 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ec1960de545a42ba9ac3e3e9adc99056: Sending DeleteTablet for 3 replicas of tablet ca167cef7420445b844d66e55a512774
I20260809 09:32:58.732112 31552 tablet_service.cc:1514] Processing DeleteTablet for tablet ca167cef7420445b844d66e55a512774 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 09:32:58 UTC) from {username='oryx', principal='oryx/127.20.125.190@KRBTEST.COM'} at 127.0.0.1:38874
I20260809 09:32:58.732196 31960 tablet_service.cc:1514] Processing DeleteTablet for tablet ca167cef7420445b844d66e55a512774 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 09:32:58 UTC) from {username='oryx', principal='oryx/127.20.125.190@KRBTEST.COM'} at 127.0.0.1:39254
I20260809 09:32:58.732746 32166 tablet_replica.cc:330] T ca167cef7420445b844d66e55a512774 P 143f334e51d54d17911a2d2ebb0b6554: stopping tablet replica
I20260809 09:32:58.732780 31765 tablet_service.cc:1514] Processing DeleteTablet for tablet ca167cef7420445b844d66e55a512774 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 09:32:58 UTC) from {username='oryx', principal='oryx/127.20.125.190@KRBTEST.COM'} at 127.0.0.1:53070
I20260809 09:32:58.733103 32169 tablet_replica.cc:330] T ca167cef7420445b844d66e55a512774 P 825ed73d552f4f8cbce972a3dc70ccd8: stopping tablet replica
I20260809 09:32:58.733413 32166 raft_consensus.cc:2229] T ca167cef7420445b844d66e55a512774 P 143f334e51d54d17911a2d2ebb0b6554 [term 1 LEADER]: Raft consensus shutting down.
I20260809 09:32:58.733445 32169 raft_consensus.cc:2229] T ca167cef7420445b844d66e55a512774 P 825ed73d552f4f8cbce972a3dc70ccd8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 09:32:58.733875 32169 raft_consensus.cc:2258] T ca167cef7420445b844d66e55a512774 P 825ed73d552f4f8cbce972a3dc70ccd8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 09:32:58.733888 32166 raft_consensus.cc:2258] T ca167cef7420445b844d66e55a512774 P 143f334e51d54d17911a2d2ebb0b6554 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 09:32:58.735576 20982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu with pid 30133
I20260809 09:32:58.742412 32166 ts_tablet_manager.cc:1905] T ca167cef7420445b844d66e55a512774 P 143f334e51d54d17911a2d2ebb0b6554: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 09:32:58.745872 32166 ts_tablet_manager.cc:1918] T ca167cef7420445b844d66e55a512774 P 143f334e51d54d17911a2d2ebb0b6554: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 09:32:58.746093 32166 log.cc:1196] T ca167cef7420445b844d66e55a512774 P 143f334e51d54d17911a2d2ebb0b6554: Deleting WAL directory at /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/minicluster-data/ts-0/wal/wals/ca167cef7420445b844d66e55a512774
I20260809 09:32:58.746660 32166 ts_tablet_manager.cc:1939] T ca167cef7420445b844d66e55a512774 P 143f334e51d54d17911a2d2ebb0b6554: Deleting consensus metadata
I20260809 09:32:58.752285 31304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:58.734553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49642 (local address 127.20.125.189:44285)
0809 09:32:58.734899 (+ 346us) server_negotiation.cc:206] Beginning negotiation
0809 09:32:58.734905 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 09:32:58.735417 (+ 512us) server_negotiation.cc:407] Connection header received
0809 09:32:58.735624 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:32:58.735628 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:32:58.735688 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:32:58.735748 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:32:58.743106 (+ 7358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:58.743989 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:58.745323 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:58.745703 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:58.745812 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:58.748125 (+ 2313us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:32:58.748129 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:32:58.748132 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:32:58.748165 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:32:58.750096 (+ 1931us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:32:58.750668 (+ 572us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:32:58.750675 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:32:58.750677 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:32:58.750796 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:32:58.751162 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:32:58.751165 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:32:58.751167 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:32:58.751430 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:32:58.751567 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0809 09:32:58.751988 (+ 421us) server_negotiation.cc:299] Negotiation successful
0809 09:32:58.752118 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":256}
I20260809 09:32:58.752954 32169 ts_tablet_manager.cc:1905] T ca167cef7420445b844d66e55a512774 P 825ed73d552f4f8cbce972a3dc70ccd8: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260809 09:32:58.753574 31490 connection.cc:535] server connection from 127.0.0.1:38874 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260809 09:32:58.756457 32169 ts_tablet_manager.cc:1918] T ca167cef7420445b844d66e55a512774 P 825ed73d552f4f8cbce972a3dc70ccd8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 09:32:58.756672 32169 log.cc:1196] T ca167cef7420445b844d66e55a512774 P 825ed73d552f4f8cbce972a3dc70ccd8: Deleting WAL directory at /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/minicluster-data/ts-2/wal/wals/ca167cef7420445b844d66e55a512774
I20260809 09:32:58.757130 32169 ts_tablet_manager.cc:1939] T ca167cef7420445b844d66e55a512774 P 825ed73d552f4f8cbce972a3dc70ccd8: Deleting consensus metadata
W20260809 09:32:58.758504 31900 connection.cc:441] server connection from 127.0.0.1:39254 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:39254 (request call id 1) could send its response
I20260809 09:32:58.758935 32172 tablet_replica.cc:330] T ca167cef7420445b844d66e55a512774 P 8bbe397caa0242579d9743094319a762: stopping tablet replica
I20260809 09:32:58.759310 32172 raft_consensus.cc:2229] T ca167cef7420445b844d66e55a512774 P 8bbe397caa0242579d9743094319a762 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 09:32:58.759666 32172 raft_consensus.cc:2258] T ca167cef7420445b844d66e55a512774 P 8bbe397caa0242579d9743094319a762 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 09:32:58.761548 32172 ts_tablet_manager.cc:1905] T ca167cef7420445b844d66e55a512774 P 8bbe397caa0242579d9743094319a762: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 09:32:58.764490 32172 ts_tablet_manager.cc:1918] T ca167cef7420445b844d66e55a512774 P 8bbe397caa0242579d9743094319a762: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 09:32:58.764596 32172 log.cc:1196] T ca167cef7420445b844d66e55a512774 P 8bbe397caa0242579d9743094319a762: Deleting WAL directory at /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/minicluster-data/ts-1/wal/wals/ca167cef7420445b844d66e55a512774
I20260809 09:32:58.764585 31669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:58.734992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37664 (local address 127.20.125.188:36019)
0809 09:32:58.735115 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0809 09:32:58.735120 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 09:32:58.753990 (+ 18870us) server_negotiation.cc:407] Connection header received
0809 09:32:58.754192 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:32:58.754197 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:32:58.754256 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:32:58.754317 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:32:58.755431 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:58.756315 (+ 884us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:58.757659 (+ 1344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:58.758012 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:58.758101 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:58.760742 (+ 2641us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:32:58.760746 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:32:58.760749 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:32:58.760784 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:32:58.762659 (+ 1875us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:32:58.763134 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:32:58.763138 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:32:58.763140 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:32:58.763237 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:32:58.763565 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:32:58.763569 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:32:58.763571 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:32:58.763812 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:32:58.763980 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0809 09:32:58.764238 (+ 258us) server_negotiation.cc:299] Negotiation successful
0809 09:32:58.764380 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260809 09:32:58.764952 32172 ts_tablet_manager.cc:1939] T ca167cef7420445b844d66e55a512774 P 8bbe397caa0242579d9743094319a762: Deleting consensus metadata
W20260809 09:32:58.766170 31697 connection.cc:441] server connection from 127.0.0.1:53070 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53070 (request call id 1) could send its response
I20260809 09:32:58.776983 31304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:58.760329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49646 (local address 127.20.125.189:44285)
0809 09:32:58.760449 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0809 09:32:58.760455 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 09:32:58.766898 (+ 6443us) server_negotiation.cc:407] Connection header received
0809 09:32:58.767116 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:32:58.767119 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:32:58.767179 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:32:58.767238 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:32:58.768407 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:58.769246 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:58.770535 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:58.770870 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:58.770988 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:58.773035 (+ 2047us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:32:58.773046 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:32:58.773049 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:32:58.773080 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:32:58.775074 (+ 1994us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:32:58.775545 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:32:58.775548 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:32:58.775550 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:32:58.775643 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:32:58.775934 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:32:58.775937 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:32:58.775939 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:32:58.776124 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:32:58.776271 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0809 09:32:58.776528 (+ 257us) server_negotiation.cc:299] Negotiation successful
0809 09:32:58.776800 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260809 09:32:58.777797 31669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:58.767031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37676 (local address 127.20.125.188:36019)
0809 09:32:58.767151 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0809 09:32:58.767155 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 09:32:58.767577 (+ 422us) server_negotiation.cc:407] Connection header received
0809 09:32:58.767689 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:32:58.767693 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:32:58.767747 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:32:58.767893 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:32:58.768767 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:58.769594 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:58.770769 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:32:58.771164 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:32:58.771279 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:32:58.773509 (+ 2230us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:32:58.773513 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:32:58.773516 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:32:58.773544 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:32:58.775644 (+ 2100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:32:58.776194 (+ 550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:32:58.776198 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:32:58.776199 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:32:58.776292 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:32:58.776585 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:32:58.776589 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:32:58.776590 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:32:58.776805 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:32:58.776946 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0809 09:32:58.777496 (+ 550us) server_negotiation.cc:299] Negotiation successful
0809 09:32:58.777626 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260809 09:32:59.719987 32184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:59.719432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:37175 (local address 127.20.125.131:59111)
0809 09:32:59.719751 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0809 09:32:59.719857 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.190:37175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":235,"thread_start_us":86,"threads_started":1}
I20260809 09:32:59.720038 32185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:59.719558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:37175 (local address 127.20.125.129:38921)
0809 09:32:59.719813 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0809 09:32:59.719890 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.190:37175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":59,"threads_started":1}
W20260809 09:32:59.720512 31658 heartbeater.cc:644] Failed to heartbeat to 127.20.125.190:37175 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.20.125.190:37175: connect: Connection refused (error 111)
W20260809 09:32:59.720527 32064 heartbeater.cc:644] Failed to heartbeat to 127.20.125.190:37175 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.20.125.190:37175: connect: Connection refused (error 111)
I20260809 09:32:59.720824 32188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:32:59.720340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:37175 (local address 127.20.125.130:57677)
0809 09:32:59.720643 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0809 09:32:59.720724 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.190:37175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":66,"threads_started":1}
W20260809 09:32:59.721062 31863 heartbeater.cc:644] Failed to heartbeat to 127.20.125.190:37175 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.20.125.190:37175: connect: Connection refused (error 111)
I20260809 09:33:00.230865 32189 raft_consensus.cc:482] T 00000000000000000000000000000000 P dece4cc043764f62a3667954979c5562 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ec1960de545a42ba9ac3e3e9adc99056)
I20260809 09:33:00.231074 32189 raft_consensus.cc:504] T 00000000000000000000000000000000 P dece4cc043764f62a3667954979c5562 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "ec1960de545a42ba9ac3e3e9adc99056" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 37175 } } peers { permanent_uuid: "3c029f42802647dea9a4c2f436be31eb" member_type: VOTER last_known_addr { host: "127.20.125.189" port: 44285 } attrs { promote: false } } peers { permanent_uuid: "dece4cc043764f62a3667954979c5562" member_type: VOTER last_known_addr { host: "127.20.125.188" port: 36019 } attrs { promote: false } }
I20260809 09:33:00.231308 32190 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3c029f42802647dea9a4c2f436be31eb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ec1960de545a42ba9ac3e3e9adc99056)
I20260809 09:33:00.231443 32190 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3c029f42802647dea9a4c2f436be31eb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "ec1960de545a42ba9ac3e3e9adc99056" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 37175 } } peers { permanent_uuid: "3c029f42802647dea9a4c2f436be31eb" member_type: VOTER last_known_addr { host: "127.20.125.189" port: 44285 } attrs { promote: false } } peers { permanent_uuid: "dece4cc043764f62a3667954979c5562" member_type: VOTER last_known_addr { host: "127.20.125.188" port: 36019 } attrs { promote: false } }
I20260809 09:33:00.232013 32190 leader_election.cc:288] T 00000000000000000000000000000000 P 3c029f42802647dea9a4c2f436be31eb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ec1960de545a42ba9ac3e3e9adc99056 (127.20.125.190:37175), dece4cc043764f62a3667954979c5562 (127.20.125.188:36019)
I20260809 09:33:00.232307 32189 leader_election.cc:288] T 00000000000000000000000000000000 P dece4cc043764f62a3667954979c5562 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ec1960de545a42ba9ac3e3e9adc99056 (127.20.125.190:37175), 3c029f42802647dea9a4c2f436be31eb (127.20.125.189:44285)
I20260809 09:33:00.232785 32191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:33:00.232238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:37175 (local address 127.0.0.1:53178)
0809 09:33:00.232572 (+ 334us) negotiation.cc:107] Waiting for socket to connect
0809 09:33:00.232666 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.190:37175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":64,"threads_started":1}
I20260809 09:33:00.232872 32192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:33:00.232385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:37175 (local address 127.0.0.1:53184)
0809 09:33:00.232681 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0809 09:33:00.232765 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.190:37175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":78,"threads_started":1}
W20260809 09:33:00.233053 31367 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.20.125.190:37175: connect: Connection refused (error 111)
W20260809 09:33:00.233112 31086 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.20.125.190:37175: connect: Connection refused (error 111)
I20260809 09:33:00.233994 32195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:33:00.233715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:37175 (local address 127.0.0.1:53186)
0809 09:33:00.233858 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0809 09:33:00.233918 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.190:37175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260809 09:33:00.234136 31367 leader_election.cc:334] T 00000000000000000000000000000000 P dece4cc043764f62a3667954979c5562 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer ec1960de545a42ba9ac3e3e9adc99056 (127.20.125.190:37175): Network error: Client connection negotiation failed: client connection to 127.20.125.190:37175: connect: Connection refused (error 111)
I20260809 09:33:00.234560 32194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:33:00.234319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:37175 (local address 127.0.0.1:53190)
0809 09:33:00.234405 (+ 86us) negotiation.cc:107] Waiting for socket to connect
0809 09:33:00.234488 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.190:37175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260809 09:33:00.234695 31086 leader_election.cc:334] T 00000000000000000000000000000000 P 3c029f42802647dea9a4c2f436be31eb [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer ec1960de545a42ba9ac3e3e9adc99056 (127.20.125.190:37175): Network error: Client connection negotiation failed: client connection to 127.20.125.190:37175: connect: Connection refused (error 111)
I20260809 09:33:00.237517 32192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:33:00.232615 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.189:44285 (local address 127.0.0.1:49650)
0809 09:33:00.233008 (+ 393us) negotiation.cc:107] Waiting for socket to connect
0809 09:33:00.233022 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 09:33:00.233135 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:33:00.233620 (+ 485us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:33:00.233626 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:33:00.233652 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 09:33:00.233796 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:33:00.233804 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:33:00.235077 (+ 1273us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:33:00.235083 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:33:00.236109 (+ 1026us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:33:00.236119 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:33:00.237001 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:33:00.237038 (+ 37us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:33:00.237191 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:33:00.237238 (+ 47us) client_negotiation.cc:769] Sending connection context
0809 09:33:00.237308 (+ 70us) client_negotiation.cc:240] Negotiation successful
0809 09:33:00.237365 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":343,"mutex_wait_us":19,"spinlock_wait_cycles":3968,"thread_start_us":59,"threads_started":1}
I20260809 09:33:00.237558 32197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:33:00.232900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49650 (local address 127.20.125.189:44285)
0809 09:33:00.233365 (+ 465us) server_negotiation.cc:206] Beginning negotiation
0809 09:33:00.233372 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 09:33:00.233388 (+ 16us) server_negotiation.cc:407] Connection header received
0809 09:33:00.233435 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:33:00.233438 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:33:00.233489 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:33:00.233913 (+ 424us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 09:33:00.234143 (+ 230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:33:00.234968 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:33:00.236246 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:33:00.236865 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:33:00.236990 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:33:00.237044 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0809 09:33:00.237323 (+ 279us) server_negotiation.cc:299] Negotiation successful
0809 09:33:00.237418 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":50,"server-negotiator.queue_time_us":385,"spinlock_wait_cycles":2048,"thread_start_us":70,"threads_started":1}
I20260809 09:33:00.237632 32191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:33:00.232407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.188:36019 (local address 127.0.0.1:37684)
0809 09:33:00.232927 (+ 520us) negotiation.cc:107] Waiting for socket to connect
0809 09:33:00.232941 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 09:33:00.233055 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:33:00.233879 (+ 824us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:33:00.233884 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:33:00.233906 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 09:33:00.234053 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:33:00.234062 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:33:00.235152 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:33:00.235156 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:33:00.236288 (+ 1132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:33:00.236297 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:33:00.237133 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:33:00.237137 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:33:00.237231 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:33:00.237251 (+ 20us) client_negotiation.cc:769] Sending connection context
0809 09:33:00.237322 (+ 71us) client_negotiation.cc:240] Negotiation successful
0809 09:33:00.237462 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":467,"mutex_wait_us":54,"spinlock_wait_cycles":2816,"thread_start_us":59,"threads_started":1}
I20260809 09:33:00.237900 32196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:33:00.232594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37684 (local address 127.20.125.188:36019)
0809 09:33:00.233565 (+ 971us) server_negotiation.cc:206] Beginning negotiation
0809 09:33:00.233570 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 09:33:00.233586 (+ 16us) server_negotiation.cc:407] Connection header received
0809 09:33:00.233648 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:33:00.233654 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:33:00.233705 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:33:00.234173 (+ 468us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 09:33:00.234270 (+ 97us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:33:00.235013 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:33:00.236411 (+ 1398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:33:00.237018 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:33:00.237145 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:33:00.237257 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0809 09:33:00.237425 (+ 168us) server_negotiation.cc:299] Negotiation successful
0809 09:33:00.237500 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":40,"server-negotiator.queue_time_us":230,"thread_start_us":77,"threads_started":1}
I20260809 09:33:00.238294 31117 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "dece4cc043764f62a3667954979c5562" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "3c029f42802647dea9a4c2f436be31eb" is_pre_election: true
I20260809 09:33:00.238349 31400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3c029f42802647dea9a4c2f436be31eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "dece4cc043764f62a3667954979c5562" is_pre_election: true
I20260809 09:33:00.238538 31117 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 3c029f42802647dea9a4c2f436be31eb [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dece4cc043764f62a3667954979c5562 in term 1.
I20260809 09:33:00.238561 31400 raft_consensus.cc:2454] T 00000000000000000000000000000000 P dece4cc043764f62a3667954979c5562 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c029f42802647dea9a4c2f436be31eb in term 1.
I20260809 09:33:00.238904 31087 leader_election.cc:302] T 00000000000000000000000000000000 P 3c029f42802647dea9a4c2f436be31eb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c029f42802647dea9a4c2f436be31eb, dece4cc043764f62a3667954979c5562; no voters: ec1960de545a42ba9ac3e3e9adc99056
I20260809 09:33:00.238966 31369 leader_election.cc:302] T 00000000000000000000000000000000 P dece4cc043764f62a3667954979c5562 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c029f42802647dea9a4c2f436be31eb, dece4cc043764f62a3667954979c5562; no voters: ec1960de545a42ba9ac3e3e9adc99056
I20260809 09:33:00.239079 32190 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3c029f42802647dea9a4c2f436be31eb [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260809 09:33:00.239142 32190 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3c029f42802647dea9a4c2f436be31eb [term 1 FOLLOWER]: Starting leader election (detected failure of leader ec1960de545a42ba9ac3e3e9adc99056)
I20260809 09:33:00.239205 32190 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3c029f42802647dea9a4c2f436be31eb [term 1 FOLLOWER]: Advancing to term 2
I20260809 09:33:00.239203 32189 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dece4cc043764f62a3667954979c5562 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260809 09:33:00.239265 32189 raft_consensus.cc:482] T 00000000000000000000000000000000 P dece4cc043764f62a3667954979c5562 [term 1 FOLLOWER]: Starting leader election (detected failure of leader ec1960de545a42ba9ac3e3e9adc99056)
I20260809 09:33:00.239312 32189 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dece4cc043764f62a3667954979c5562 [term 1 FOLLOWER]: Advancing to term 2
I20260809 09:33:00.240265 32190 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3c029f42802647dea9a4c2f436be31eb [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "ec1960de545a42ba9ac3e3e9adc99056" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 37175 } } peers { permanent_uuid: "3c029f42802647dea9a4c2f436be31eb" member_type: VOTER last_known_addr { host: "127.20.125.189" port: 44285 } attrs { promote: false } } peers { permanent_uuid: "dece4cc043764f62a3667954979c5562" member_type: VOTER last_known_addr { host: "127.20.125.188" port: 36019 } attrs { promote: false } }
I20260809 09:33:00.240319 32189 raft_consensus.cc:504] T 00000000000000000000000000000000 P dece4cc043764f62a3667954979c5562 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "ec1960de545a42ba9ac3e3e9adc99056" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 37175 } } peers { permanent_uuid: "3c029f42802647dea9a4c2f436be31eb" member_type: VOTER last_known_addr { host: "127.20.125.189" port: 44285 } attrs { promote: false } } peers { permanent_uuid: "dece4cc043764f62a3667954979c5562" member_type: VOTER last_known_addr { host: "127.20.125.188" port: 36019 } attrs { promote: false } }
I20260809 09:33:00.240645 32190 leader_election.cc:288] T 00000000000000000000000000000000 P 3c029f42802647dea9a4c2f436be31eb [CANDIDATE]: Term 2 election: Requested vote from peers ec1960de545a42ba9ac3e3e9adc99056 (127.20.125.190:37175), dece4cc043764f62a3667954979c5562 (127.20.125.188:36019)
I20260809 09:33:00.240654 32189 leader_election.cc:288] T 00000000000000000000000000000000 P dece4cc043764f62a3667954979c5562 [CANDIDATE]: Term 2 election: Requested vote from peers ec1960de545a42ba9ac3e3e9adc99056 (127.20.125.190:37175), 3c029f42802647dea9a4c2f436be31eb (127.20.125.189:44285)
I20260809 09:33:00.241057 32192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:33:00.240761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:37175 (local address 127.0.0.1:53210)
0809 09:33:00.240909 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0809 09:33:00.240954 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.190:37175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260809 09:33:00.241153 31400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3c029f42802647dea9a4c2f436be31eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "dece4cc043764f62a3667954979c5562"
I20260809 09:33:00.241307 31400 raft_consensus.cc:2379] T 00000000000000000000000000000000 P dece4cc043764f62a3667954979c5562 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c029f42802647dea9a4c2f436be31eb in current term 2: Already voted for candidate dece4cc043764f62a3667954979c5562 in this term.
I20260809 09:33:00.241349 31117 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "dece4cc043764f62a3667954979c5562" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "3c029f42802647dea9a4c2f436be31eb"
I20260809 09:33:00.241494 31117 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 3c029f42802647dea9a4c2f436be31eb [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate dece4cc043764f62a3667954979c5562 in current term 2: Already voted for candidate 3c029f42802647dea9a4c2f436be31eb in this term.
I20260809 09:33:00.241735 32192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:33:00.241496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:37175 (local address 127.0.0.1:53232)
0809 09:33:00.241609 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0809 09:33:00.241666 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.190:37175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260809 09:33:00.241878 32191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:33:00.241094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:37175 (local address 127.0.0.1:53218)
0809 09:33:00.241753 (+ 659us) negotiation.cc:107] Waiting for socket to connect
0809 09:33:00.241812 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.190:37175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":591}
W20260809 09:33:00.242055 31367 leader_election.cc:334] T 00000000000000000000000000000000 P dece4cc043764f62a3667954979c5562 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer ec1960de545a42ba9ac3e3e9adc99056 (127.20.125.190:37175): Network error: Client connection negotiation failed: client connection to 127.20.125.190:37175: connect: Connection refused (error 111)
I20260809 09:33:00.242134 31367 leader_election.cc:302] T 00000000000000000000000000000000 P dece4cc043764f62a3667954979c5562 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dece4cc043764f62a3667954979c5562; no voters: 3c029f42802647dea9a4c2f436be31eb, ec1960de545a42ba9ac3e3e9adc99056
I20260809 09:33:00.242280 32189 raft_consensus.cc:2726] T 00000000000000000000000000000000 P dece4cc043764f62a3667954979c5562 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260809 09:33:00.242635 32191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:33:00.242424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:37175 (local address 127.0.0.1:53246)
0809 09:33:00.242533 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0809 09:33:00.242587 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.190:37175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260809 09:33:00.242792 31086 leader_election.cc:334] T 00000000000000000000000000000000 P 3c029f42802647dea9a4c2f436be31eb [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer ec1960de545a42ba9ac3e3e9adc99056 (127.20.125.190:37175): Network error: Client connection negotiation failed: client connection to 127.20.125.190:37175: connect: Connection refused (error 111)
I20260809 09:33:00.242859 31086 leader_election.cc:302] T 00000000000000000000000000000000 P 3c029f42802647dea9a4c2f436be31eb [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c029f42802647dea9a4c2f436be31eb; no voters: dece4cc043764f62a3667954979c5562, ec1960de545a42ba9ac3e3e9adc99056
I20260809 09:33:00.242983 32190 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 3c029f42802647dea9a4c2f436be31eb [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260809 09:33:00.721721 32202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:33:00.721096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:37175 (local address 127.20.125.131:51731)
0809 09:33:00.721485 (+ 389us) negotiation.cc:107] Waiting for socket to connect
0809 09:33:00.721568 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.190:37175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":303,"thread_start_us":155,"threads_started":1}
I20260809 09:33:00.721837 32203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:33:00.721097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:37175 (local address 127.20.125.129:44965)
0809 09:33:00.721659 (+ 562us) negotiation.cc:107] Waiting for socket to connect
0809 09:33:00.721724 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.190:37175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":475,"thread_start_us":156,"threads_started":1}
I20260809 09:33:00.722038 32205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:33:00.721591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:37175 (local address 127.20.125.130:35005)
0809 09:33:00.721887 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0809 09:33:00.721944 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.190:37175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":85,"threads_started":1}
I20260809 09:33:01.723043 32206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:33:01.722541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:37175 (local address 127.20.125.131:55263)
0809 09:33:01.722855 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0809 09:33:01.722925 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.190:37175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":85,"threads_started":1}
I20260809 09:33:01.723078 32207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:33:01.722577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:37175 (local address 127.20.125.130:33683)
0809 09:33:01.722937 (+ 360us) negotiation.cc:107] Waiting for socket to connect
0809 09:33:01.722989 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.190:37175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":291,"thread_start_us":62,"threads_started":1}
I20260809 09:33:01.723068 32210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:33:01.722548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:37175 (local address 127.20.125.129:48273)
0809 09:33:01.722903 (+ 355us) negotiation.cc:107] Waiting for socket to connect
0809 09:33:01.722966 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.190:37175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":238,"thread_start_us":76,"threads_started":1}
I20260809 09:33:02.013890 32212 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3c029f42802647dea9a4c2f436be31eb [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 09:33:02.014055 32212 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3c029f42802647dea9a4c2f436be31eb [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "ec1960de545a42ba9ac3e3e9adc99056" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 37175 } } peers { permanent_uuid: "3c029f42802647dea9a4c2f436be31eb" member_type: VOTER last_known_addr { host: "127.20.125.189" port: 44285 } attrs { promote: false } } peers { permanent_uuid: "dece4cc043764f62a3667954979c5562" member_type: VOTER last_known_addr { host: "127.20.125.188" port: 36019 } attrs { promote: false } }
I20260809 09:33:02.014778 32212 leader_election.cc:288] T 00000000000000000000000000000000 P 3c029f42802647dea9a4c2f436be31eb [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ec1960de545a42ba9ac3e3e9adc99056 (127.20.125.190:37175), dece4cc043764f62a3667954979c5562 (127.20.125.188:36019)
I20260809 09:33:02.015350 31400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3c029f42802647dea9a4c2f436be31eb" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "dece4cc043764f62a3667954979c5562" is_pre_election: true
I20260809 09:33:02.015477 32214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:33:02.014987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:37175 (local address 127.0.0.1:53500)
0809 09:33:02.015320 (+ 333us) negotiation.cc:107] Waiting for socket to connect
0809 09:33:02.015386 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.190:37175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":86,"threads_started":1}
I20260809 09:33:02.015579 31400 raft_consensus.cc:2454] T 00000000000000000000000000000000 P dece4cc043764f62a3667954979c5562 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c029f42802647dea9a4c2f436be31eb in term 2.
I20260809 09:33:02.015959 31087 leader_election.cc:302] T 00000000000000000000000000000000 P 3c029f42802647dea9a4c2f436be31eb [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c029f42802647dea9a4c2f436be31eb, dece4cc043764f62a3667954979c5562; no voters:
I20260809 09:33:02.016247 32212 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3c029f42802647dea9a4c2f436be31eb [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260809 09:33:02.016323 32212 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3c029f42802647dea9a4c2f436be31eb [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 09:33:02.016371 32212 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3c029f42802647dea9a4c2f436be31eb [term 2 FOLLOWER]: Advancing to term 3
I20260809 09:33:02.016474 32214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:33:02.016205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:37175 (local address 127.0.0.1:53516)
0809 09:33:02.016334 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0809 09:33:02.016398 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.190:37175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260809 09:33:02.016651 31086 leader_election.cc:334] T 00000000000000000000000000000000 P 3c029f42802647dea9a4c2f436be31eb [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer ec1960de545a42ba9ac3e3e9adc99056 (127.20.125.190:37175): Network error: Client connection negotiation failed: client connection to 127.20.125.190:37175: connect: Connection refused (error 111)
I20260809 09:33:02.017539 32212 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3c029f42802647dea9a4c2f436be31eb [term 3 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "ec1960de545a42ba9ac3e3e9adc99056" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 37175 } } peers { permanent_uuid: "3c029f42802647dea9a4c2f436be31eb" member_type: VOTER last_known_addr { host: "127.20.125.189" port: 44285 } attrs { promote: false } } peers { permanent_uuid: "dece4cc043764f62a3667954979c5562" member_type: VOTER last_known_addr { host: "127.20.125.188" port: 36019 } attrs { promote: false } }
I20260809 09:33:02.017879 32212 leader_election.cc:288] T 00000000000000000000000000000000 P 3c029f42802647dea9a4c2f436be31eb [CANDIDATE]: Term 3 election: Requested vote from peers ec1960de545a42ba9ac3e3e9adc99056 (127.20.125.190:37175), dece4cc043764f62a3667954979c5562 (127.20.125.188:36019)
I20260809 09:33:02.018256 31400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3c029f42802647dea9a4c2f436be31eb" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "dece4cc043764f62a3667954979c5562"
I20260809 09:33:02.018302 32214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:33:02.018058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:37175 (local address 127.0.0.1:53532)
0809 09:33:02.018188 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0809 09:33:02.018243 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.190:37175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260809 09:33:02.018391 31400 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dece4cc043764f62a3667954979c5562 [term 2 FOLLOWER]: Advancing to term 3
I20260809 09:33:02.019094 32214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:33:02.018864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:37175 (local address 127.0.0.1:53544)
0809 09:33:02.018972 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0809 09:33:02.019038 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.190:37175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260809 09:33:02.019217 31086 leader_election.cc:334] T 00000000000000000000000000000000 P 3c029f42802647dea9a4c2f436be31eb [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer ec1960de545a42ba9ac3e3e9adc99056 (127.20.125.190:37175): Network error: Client connection negotiation failed: client connection to 127.20.125.190:37175: connect: Connection refused (error 111)
I20260809 09:33:02.019621 31400 raft_consensus.cc:2454] T 00000000000000000000000000000000 P dece4cc043764f62a3667954979c5562 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c029f42802647dea9a4c2f436be31eb in term 3.
I20260809 09:33:02.019929 31087 leader_election.cc:302] T 00000000000000000000000000000000 P 3c029f42802647dea9a4c2f436be31eb [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c029f42802647dea9a4c2f436be31eb, dece4cc043764f62a3667954979c5562; no voters: ec1960de545a42ba9ac3e3e9adc99056
I20260809 09:33:02.020201 32212 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3c029f42802647dea9a4c2f436be31eb [term 3 FOLLOWER]: Leader election won for term 3
I20260809 09:33:02.020414 32212 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3c029f42802647dea9a4c2f436be31eb [term 3 LEADER]: Becoming Leader. State: Replica: 3c029f42802647dea9a4c2f436be31eb, State: Running, Role: LEADER
I20260809 09:33:02.020732 32212 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3c029f42802647dea9a4c2f436be31eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "ec1960de545a42ba9ac3e3e9adc99056" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 37175 } } peers { permanent_uuid: "3c029f42802647dea9a4c2f436be31eb" member_type: VOTER last_known_addr { host: "127.20.125.189" port: 44285 } attrs { promote: false } } peers { permanent_uuid: "dece4cc043764f62a3667954979c5562" member_type: VOTER last_known_addr { host: "127.20.125.188" port: 36019 } attrs { promote: false } }
I20260809 09:33:02.021930 32216 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3c029f42802647dea9a4c2f436be31eb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3c029f42802647dea9a4c2f436be31eb. Latest consensus state: current_term: 3 leader_uuid: "3c029f42802647dea9a4c2f436be31eb" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "ec1960de545a42ba9ac3e3e9adc99056" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 37175 } } peers { permanent_uuid: "3c029f42802647dea9a4c2f436be31eb" member_type: VOTER last_known_addr { host: "127.20.125.189" port: 44285 } attrs { promote: false } } peers { permanent_uuid: "dece4cc043764f62a3667954979c5562" member_type: VOTER last_known_addr { host: "127.20.125.188" port: 36019 } attrs { promote: false } } }
I20260809 09:33:02.022063 32216 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3c029f42802647dea9a4c2f436be31eb [sys.catalog]: This master's current role is: LEADER
I20260809 09:33:02.022372 32218 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 09:33:02.025002 32218 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 09:33:02.025449 32218 catalog_manager.cc:1259] Loaded cluster ID: a876011cffd24bdd84e8602f372dcb24
I20260809 09:33:02.025501 32218 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 09:33:02.026125 32218 catalog_manager.cc:1504] Loading token signing keys...
I20260809 09:33:02.026623 32218 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 3c029f42802647dea9a4c2f436be31eb: Loaded TSK: 0
I20260809 09:33:02.027343 32218 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 09:33:02.080570 31112 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49646:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 09:33:02.083719 31112 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 09:33:02.086928 32214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:33:02.086613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:37175 (local address 127.0.0.1:53556)
0809 09:33:02.086782 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0809 09:33:02.086844 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.190:37175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260809 09:33:02.086941 31400 raft_consensus.cc:1261] T 00000000000000000000000000000000 P dece4cc043764f62a3667954979c5562 [term 3 FOLLOWER]: Refusing update from remote peer 3c029f42802647dea9a4c2f436be31eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 3 index: 14. (index mismatch)
I20260809 09:33:02.087430 32212 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 3c029f42802647dea9a4c2f436be31eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "dece4cc043764f62a3667954979c5562" member_type: VOTER last_known_addr { host: "127.20.125.188" port: 36019 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260809 09:33:02.087841 32214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:33:02.087504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:37175 (local address 127.0.0.1:53572)
0809 09:33:02.087693 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0809 09:33:02.087757 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.190:37175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94}
W20260809 09:33:02.087981 31086 consensus_peers.cc:487] T 00000000000000000000000000000000 P 3c029f42802647dea9a4c2f436be31eb -> Peer ec1960de545a42ba9ac3e3e9adc99056 (127.20.125.190:37175): Couldn't send request to peer ec1960de545a42ba9ac3e3e9adc99056. Status: Network error: Client connection negotiation failed: client connection to 127.20.125.190:37175: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260809 09:33:02.089104 32221 sys_catalog.cc:455] T 00000000000000000000000000000000 P dece4cc043764f62a3667954979c5562 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3c029f42802647dea9a4c2f436be31eb. Latest consensus state: current_term: 3 leader_uuid: "3c029f42802647dea9a4c2f436be31eb" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "ec1960de545a42ba9ac3e3e9adc99056" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 37175 } } peers { permanent_uuid: "3c029f42802647dea9a4c2f436be31eb" member_type: VOTER last_known_addr { host: "127.20.125.189" port: 44285 } attrs { promote: false } } peers { permanent_uuid: "dece4cc043764f62a3667954979c5562" member_type: VOTER last_known_addr { host: "127.20.125.188" port: 36019 } attrs { promote: false } } }
I20260809 09:33:02.089250 32221 sys_catalog.cc:458] T 00000000000000000000000000000000 P dece4cc043764f62a3667954979c5562 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 09:33:02.090649 32212 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3c029f42802647dea9a4c2f436be31eb [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "3c029f42802647dea9a4c2f436be31eb" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "ec1960de545a42ba9ac3e3e9adc99056" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 37175 } } peers { permanent_uuid: "3c029f42802647dea9a4c2f436be31eb" member_type: VOTER last_known_addr { host: "127.20.125.189" port: 44285 } attrs { promote: false } } peers { permanent_uuid: "dece4cc043764f62a3667954979c5562" member_type: VOTER last_known_addr { host: "127.20.125.188" port: 36019 } attrs { promote: false } } }
I20260809 09:33:02.090806 32212 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3c029f42802647dea9a4c2f436be31eb [sys.catalog]: This master's current role is: LEADER
I20260809 09:33:02.091280 32221 sys_catalog.cc:455] T 00000000000000000000000000000000 P dece4cc043764f62a3667954979c5562 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "3c029f42802647dea9a4c2f436be31eb" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "ec1960de545a42ba9ac3e3e9adc99056" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 37175 } } peers { permanent_uuid: "3c029f42802647dea9a4c2f436be31eb" member_type: VOTER last_known_addr { host: "127.20.125.189" port: 44285 } attrs { promote: false } } peers { permanent_uuid: "dece4cc043764f62a3667954979c5562" member_type: VOTER last_known_addr { host: "127.20.125.188" port: 36019 } attrs { promote: false } } }
I20260809 09:33:02.091457 32221 sys_catalog.cc:458] T 00000000000000000000000000000000 P dece4cc043764f62a3667954979c5562 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 09:33:02.101997 32214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:33:02.096991 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.130:44297 (local address 127.0.0.1:53080)
0809 09:33:02.097143 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0809 09:33:02.097161 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 09:33:02.097286 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:33:02.098112 (+ 826us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:33:02.098116 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:33:02.098136 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 09:33:02.098269 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:33:02.098276 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:33:02.099491 (+ 1215us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:33:02.099494 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:33:02.100741 (+ 1247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:33:02.100749 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:33:02.101567 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:33:02.101570 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:33:02.101662 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:33:02.101686 (+ 24us) client_negotiation.cc:769] Sending connection context
0809 09:33:02.101785 (+ 99us) client_negotiation.cc:240] Negotiation successful
0809 09:33:02.101859 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":78,"mutex_wait_us":78}
I20260809 09:33:02.102013 32228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:33:02.097233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53080 (local address 127.20.125.130:44297)
0809 09:33:02.097819 (+ 586us) server_negotiation.cc:206] Beginning negotiation
0809 09:33:02.097825 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 09:33:02.097843 (+ 18us) server_negotiation.cc:407] Connection header received
0809 09:33:02.097904 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:33:02.097908 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:33:02.097967 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:33:02.098392 (+ 425us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 09:33:02.098559 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:33:02.099353 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:33:02.100860 (+ 1507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:33:02.101448 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:33:02.101582 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:33:02.101638 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0809 09:33:02.101803 (+ 165us) server_negotiation.cc:299] Negotiation successful
0809 09:33:02.101877 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":479,"thread_start_us":80,"threads_started":1}
I20260809 09:33:02.102667 31765 tablet_service.cc:1467] Processing CreateTablet for tablet ff306e9576134d12a3f3c5fec97445d8 (DEFAULT_TABLE table=test-table [id=1767962b94bf4795a5f6f9bb7cf91ee8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 09:33:02.102815 32229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:33:02.097273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.131:44279 (local address 127.0.0.1:39256)
0809 09:33:02.097542 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0809 09:33:02.097557 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 09:33:02.097645 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:33:02.098235 (+ 590us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:33:02.098238 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:33:02.098252 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 09:33:02.098348 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:33:02.098358 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:33:02.099364 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:33:02.099368 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:33:02.100785 (+ 1417us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:33:02.100795 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:33:02.102336 (+ 1541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:33:02.102339 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:33:02.102435 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:33:02.102497 (+ 62us) client_negotiation.cc:769] Sending connection context
0809 09:33:02.102574 (+ 77us) client_negotiation.cc:240] Negotiation successful
0809 09:33:02.102647 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":200,"mutex_wait_us":7,"thread_start_us":127,"threads_started":1}
I20260809 09:33:02.102831 32231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:33:02.097682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39256 (local address 127.20.125.131:44279)
0809 09:33:02.097964 (+ 282us) server_negotiation.cc:206] Beginning negotiation
0809 09:33:02.097969 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 09:33:02.097985 (+ 16us) server_negotiation.cc:407] Connection header received
0809 09:33:02.098039 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:33:02.098045 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:33:02.098104 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:33:02.098229 (+ 125us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 09:33:02.098498 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:33:02.099247 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:33:02.100940 (+ 1693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:33:02.101635 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:33:02.101757 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:33:02.101817 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0809 09:33:02.102613 (+ 796us) server_negotiation.cc:299] Negotiation successful
0809 09:33:02.102694 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":198,"thread_start_us":90,"threads_started":1}
I20260809 09:33:02.103021 31765 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ff306e9576134d12a3f3c5fec97445d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 09:33:02.103318 32232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:33:02.097697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.129:42545 (local address 127.0.0.1:38890)
0809 09:33:02.097995 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0809 09:33:02.098013 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 09:33:02.098102 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:33:02.099643 (+ 1541us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:33:02.099647 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:33:02.099663 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 09:33:02.099753 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:33:02.099759 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:33:02.100490 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:33:02.100493 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:33:02.101768 (+ 1275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:33:02.101778 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:33:02.102882 (+ 1104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:33:02.102888 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:33:02.103011 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:33:02.103039 (+ 28us) client_negotiation.cc:769] Sending connection context
0809 09:33:02.103117 (+ 78us) client_negotiation.cc:240] Negotiation successful
0809 09:33:02.103189 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":229,"thread_start_us":139,"threads_started":1}
I20260809 09:33:02.103382 32233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:33:02.098388 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38890 (local address 127.20.125.129:42545)
0809 09:33:02.099440 (+ 1052us) server_negotiation.cc:206] Beginning negotiation
0809 09:33:02.099443 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 09:33:02.099454 (+ 11us) server_negotiation.cc:407] Connection header received
0809 09:33:02.099496 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:33:02.099499 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:33:02.099536 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:33:02.099634 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 09:33:02.099869 (+ 235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:33:02.100372 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:33:02.101919 (+ 1547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:33:02.102632 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:33:02.102763 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:33:02.102813 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0809 09:33:02.103152 (+ 339us) server_negotiation.cc:299] Negotiation successful
0809 09:33:02.103247 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":979,"thread_start_us":106,"threads_started":1}
I20260809 09:33:02.103353 31960 tablet_service.cc:1467] Processing CreateTablet for tablet ff306e9576134d12a3f3c5fec97445d8 (DEFAULT_TABLE table=test-table [id=1767962b94bf4795a5f6f9bb7cf91ee8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 09:33:02.103631 31960 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ff306e9576134d12a3f3c5fec97445d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 09:33:02.103883 31552 tablet_service.cc:1467] Processing CreateTablet for tablet ff306e9576134d12a3f3c5fec97445d8 (DEFAULT_TABLE table=test-table [id=1767962b94bf4795a5f6f9bb7cf91ee8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 09:33:02.104111 31552 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ff306e9576134d12a3f3c5fec97445d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 09:33:02.106215 32235 tablet_bootstrap.cc:492] T ff306e9576134d12a3f3c5fec97445d8 P 825ed73d552f4f8cbce972a3dc70ccd8: Bootstrap starting.
I20260809 09:33:02.106211 32234 tablet_bootstrap.cc:492] T ff306e9576134d12a3f3c5fec97445d8 P 8bbe397caa0242579d9743094319a762: Bootstrap starting.
I20260809 09:33:02.106927 32236 tablet_bootstrap.cc:492] T ff306e9576134d12a3f3c5fec97445d8 P 143f334e51d54d17911a2d2ebb0b6554: Bootstrap starting.
I20260809 09:33:02.107266 32234 tablet_bootstrap.cc:654] T ff306e9576134d12a3f3c5fec97445d8 P 8bbe397caa0242579d9743094319a762: Neither blocks nor log segments found. Creating new log.
I20260809 09:33:02.107328 32235 tablet_bootstrap.cc:654] T ff306e9576134d12a3f3c5fec97445d8 P 825ed73d552f4f8cbce972a3dc70ccd8: Neither blocks nor log segments found. Creating new log.
I20260809 09:33:02.108071 32236 tablet_bootstrap.cc:654] T ff306e9576134d12a3f3c5fec97445d8 P 143f334e51d54d17911a2d2ebb0b6554: Neither blocks nor log segments found. Creating new log.
I20260809 09:33:02.108470 32234 tablet_bootstrap.cc:492] T ff306e9576134d12a3f3c5fec97445d8 P 8bbe397caa0242579d9743094319a762: No bootstrap required, opened a new log
I20260809 09:33:02.108580 32234 ts_tablet_manager.cc:1397] T ff306e9576134d12a3f3c5fec97445d8 P 8bbe397caa0242579d9743094319a762: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 09:33:02.109157 32234 raft_consensus.cc:348] T ff306e9576134d12a3f3c5fec97445d8 P 8bbe397caa0242579d9743094319a762 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bbe397caa0242579d9743094319a762" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 44297 } } peers { permanent_uuid: "825ed73d552f4f8cbce972a3dc70ccd8" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 44279 } } peers { permanent_uuid: "143f334e51d54d17911a2d2ebb0b6554" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 42545 } }
I20260809 09:33:02.109274 32236 tablet_bootstrap.cc:492] T ff306e9576134d12a3f3c5fec97445d8 P 143f334e51d54d17911a2d2ebb0b6554: No bootstrap required, opened a new log
I20260809 09:33:02.109287 32234 raft_consensus.cc:374] T ff306e9576134d12a3f3c5fec97445d8 P 8bbe397caa0242579d9743094319a762 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 09:33:02.109325 32234 raft_consensus.cc:729] T ff306e9576134d12a3f3c5fec97445d8 P 8bbe397caa0242579d9743094319a762 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8bbe397caa0242579d9743094319a762, State: Initialized, Role: FOLLOWER
I20260809 09:33:02.109369 32236 ts_tablet_manager.cc:1397] T ff306e9576134d12a3f3c5fec97445d8 P 143f334e51d54d17911a2d2ebb0b6554: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 09:33:02.109519 32235 tablet_bootstrap.cc:492] T ff306e9576134d12a3f3c5fec97445d8 P 825ed73d552f4f8cbce972a3dc70ccd8: No bootstrap required, opened a new log
I20260809 09:33:02.109475 32234 consensus_queue.cc:260] T ff306e9576134d12a3f3c5fec97445d8 P 8bbe397caa0242579d9743094319a762 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bbe397caa0242579d9743094319a762" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 44297 } } peers { permanent_uuid: "825ed73d552f4f8cbce972a3dc70ccd8" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 44279 } } peers { permanent_uuid: "143f334e51d54d17911a2d2ebb0b6554" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 42545 } }
I20260809 09:33:02.109676 32235 ts_tablet_manager.cc:1397] T ff306e9576134d12a3f3c5fec97445d8 P 825ed73d552f4f8cbce972a3dc70ccd8: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260809 09:33:02.109889 32234 ts_tablet_manager.cc:1428] T ff306e9576134d12a3f3c5fec97445d8 P 8bbe397caa0242579d9743094319a762: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260809 09:33:02.109898 32236 raft_consensus.cc:348] T ff306e9576134d12a3f3c5fec97445d8 P 143f334e51d54d17911a2d2ebb0b6554 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bbe397caa0242579d9743094319a762" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 44297 } } peers { permanent_uuid: "825ed73d552f4f8cbce972a3dc70ccd8" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 44279 } } peers { permanent_uuid: "143f334e51d54d17911a2d2ebb0b6554" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 42545 } }
I20260809 09:33:02.110028 32236 raft_consensus.cc:374] T ff306e9576134d12a3f3c5fec97445d8 P 143f334e51d54d17911a2d2ebb0b6554 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 09:33:02.110074 32236 raft_consensus.cc:729] T ff306e9576134d12a3f3c5fec97445d8 P 143f334e51d54d17911a2d2ebb0b6554 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 143f334e51d54d17911a2d2ebb0b6554, State: Initialized, Role: FOLLOWER
I20260809 09:33:02.110208 32236 consensus_queue.cc:260] T ff306e9576134d12a3f3c5fec97445d8 P 143f334e51d54d17911a2d2ebb0b6554 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bbe397caa0242579d9743094319a762" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 44297 } } peers { permanent_uuid: "825ed73d552f4f8cbce972a3dc70ccd8" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 44279 } } peers { permanent_uuid: "143f334e51d54d17911a2d2ebb0b6554" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 42545 } }
I20260809 09:33:02.110437 32235 raft_consensus.cc:348] T ff306e9576134d12a3f3c5fec97445d8 P 825ed73d552f4f8cbce972a3dc70ccd8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bbe397caa0242579d9743094319a762" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 44297 } } peers { permanent_uuid: "825ed73d552f4f8cbce972a3dc70ccd8" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 44279 } } peers { permanent_uuid: "143f334e51d54d17911a2d2ebb0b6554" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 42545 } }
I20260809 09:33:02.110627 32207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:33:02.110299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:37175 (local address 127.20.125.130:55067)
0809 09:33:02.110489 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0809 09:33:02.110547 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.190:37175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92}
I20260809 09:33:02.110656 32236 ts_tablet_manager.cc:1428] T ff306e9576134d12a3f3c5fec97445d8 P 143f334e51d54d17911a2d2ebb0b6554: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260809 09:33:02.110625 32235 raft_consensus.cc:374] T ff306e9576134d12a3f3c5fec97445d8 P 825ed73d552f4f8cbce972a3dc70ccd8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 09:33:02.110870 32235 raft_consensus.cc:729] T ff306e9576134d12a3f3c5fec97445d8 P 825ed73d552f4f8cbce972a3dc70ccd8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 825ed73d552f4f8cbce972a3dc70ccd8, State: Initialized, Role: FOLLOWER
I20260809 09:33:02.111056 32235 consensus_queue.cc:260] T ff306e9576134d12a3f3c5fec97445d8 P 825ed73d552f4f8cbce972a3dc70ccd8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bbe397caa0242579d9743094319a762" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 44297 } } peers { permanent_uuid: "825ed73d552f4f8cbce972a3dc70ccd8" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 44279 } } peers { permanent_uuid: "143f334e51d54d17911a2d2ebb0b6554" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 42545 } }
I20260809 09:33:02.111479 32235 ts_tablet_manager.cc:1428] T ff306e9576134d12a3f3c5fec97445d8 P 825ed73d552f4f8cbce972a3dc70ccd8: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260809 09:33:02.112418 32210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:33:02.111858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:37175 (local address 127.20.125.129:41723)
0809 09:33:02.112271 (+ 413us) negotiation.cc:107] Waiting for socket to connect
0809 09:33:02.112338 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.190:37175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":321}
I20260809 09:33:02.113075 32207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:33:02.112779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:37175 (local address 127.20.125.130:38645)
0809 09:33:02.112939 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0809 09:33:02.112994 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.190:37175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
I20260809 09:33:02.113463 32210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:33:02.113188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:37175 (local address 127.20.125.129:39937)
0809 09:33:02.113344 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0809 09:33:02.113397 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.190:37175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86}
I20260809 09:33:02.113705 31864 heartbeater.cc:503] Master 127.20.125.189:44285 was elected leader, sending a full tablet report...
I20260809 09:33:02.114291 32206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:33:02.113852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:37175 (local address 127.20.125.131:48889)
0809 09:33:02.114125 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0809 09:33:02.114203 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.190:37175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":190}
I20260809 09:33:02.114770 31660 heartbeater.cc:503] Master 127.20.125.189:44285 was elected leader, sending a full tablet report...
I20260809 09:33:02.115793 32068 heartbeater.cc:503] Master 127.20.125.189:44285 was elected leader, sending a full tablet report...
I20260809 09:33:02.116041 32206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:33:02.115740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:37175 (local address 127.20.125.131:60025)
0809 09:33:02.115908 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0809 09:33:02.115959 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.190:37175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92}
I20260809 09:33:02.184520 32242 raft_consensus.cc:482] T ff306e9576134d12a3f3c5fec97445d8 P 143f334e51d54d17911a2d2ebb0b6554 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 09:33:02.184682 32242 raft_consensus.cc:504] T ff306e9576134d12a3f3c5fec97445d8 P 143f334e51d54d17911a2d2ebb0b6554 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bbe397caa0242579d9743094319a762" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 44297 } } peers { permanent_uuid: "825ed73d552f4f8cbce972a3dc70ccd8" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 44279 } } peers { permanent_uuid: "143f334e51d54d17911a2d2ebb0b6554" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 42545 } }
I20260809 09:33:02.185058 32242 leader_election.cc:288] T ff306e9576134d12a3f3c5fec97445d8 P 143f334e51d54d17911a2d2ebb0b6554 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8bbe397caa0242579d9743094319a762 (127.20.125.130:44297), 825ed73d552f4f8cbce972a3dc70ccd8 (127.20.125.131:44279)
I20260809 09:33:02.185570 31994 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ff306e9576134d12a3f3c5fec97445d8" candidate_uuid: "143f334e51d54d17911a2d2ebb0b6554" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "825ed73d552f4f8cbce972a3dc70ccd8" is_pre_election: true
I20260809 09:33:02.185570 31793 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ff306e9576134d12a3f3c5fec97445d8" candidate_uuid: "143f334e51d54d17911a2d2ebb0b6554" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8bbe397caa0242579d9743094319a762" is_pre_election: true
I20260809 09:33:02.185750 31994 raft_consensus.cc:2454] T ff306e9576134d12a3f3c5fec97445d8 P 825ed73d552f4f8cbce972a3dc70ccd8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 143f334e51d54d17911a2d2ebb0b6554 in term 0.
I20260809 09:33:02.185763 31793 raft_consensus.cc:2454] T ff306e9576134d12a3f3c5fec97445d8 P 8bbe397caa0242579d9743094319a762 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 143f334e51d54d17911a2d2ebb0b6554 in term 0.
I20260809 09:33:02.186131 31493 leader_election.cc:302] T ff306e9576134d12a3f3c5fec97445d8 P 143f334e51d54d17911a2d2ebb0b6554 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 143f334e51d54d17911a2d2ebb0b6554, 825ed73d552f4f8cbce972a3dc70ccd8, 8bbe397caa0242579d9743094319a762; no voters:
I20260809 09:33:02.186378 32242 raft_consensus.cc:2781] T ff306e9576134d12a3f3c5fec97445d8 P 143f334e51d54d17911a2d2ebb0b6554 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 09:33:02.186510 32242 raft_consensus.cc:482] T ff306e9576134d12a3f3c5fec97445d8 P 143f334e51d54d17911a2d2ebb0b6554 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 09:33:02.186573 32242 raft_consensus.cc:3037] T ff306e9576134d12a3f3c5fec97445d8 P 143f334e51d54d17911a2d2ebb0b6554 [term 0 FOLLOWER]: Advancing to term 1
I20260809 09:33:02.187755 32242 raft_consensus.cc:504] T ff306e9576134d12a3f3c5fec97445d8 P 143f334e51d54d17911a2d2ebb0b6554 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bbe397caa0242579d9743094319a762" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 44297 } } peers { permanent_uuid: "825ed73d552f4f8cbce972a3dc70ccd8" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 44279 } } peers { permanent_uuid: "143f334e51d54d17911a2d2ebb0b6554" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 42545 } }
I20260809 09:33:02.188181 32242 leader_election.cc:288] T ff306e9576134d12a3f3c5fec97445d8 P 143f334e51d54d17911a2d2ebb0b6554 [CANDIDATE]: Term 1 election: Requested vote from peers 8bbe397caa0242579d9743094319a762 (127.20.125.130:44297), 825ed73d552f4f8cbce972a3dc70ccd8 (127.20.125.131:44279)
I20260809 09:33:02.188556 31994 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ff306e9576134d12a3f3c5fec97445d8" candidate_uuid: "143f334e51d54d17911a2d2ebb0b6554" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "825ed73d552f4f8cbce972a3dc70ccd8"
I20260809 09:33:02.188692 31994 raft_consensus.cc:3037] T ff306e9576134d12a3f3c5fec97445d8 P 825ed73d552f4f8cbce972a3dc70ccd8 [term 0 FOLLOWER]: Advancing to term 1
I20260809 09:33:02.188640 31793 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ff306e9576134d12a3f3c5fec97445d8" candidate_uuid: "143f334e51d54d17911a2d2ebb0b6554" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8bbe397caa0242579d9743094319a762"
I20260809 09:33:02.188822 31793 raft_consensus.cc:3037] T ff306e9576134d12a3f3c5fec97445d8 P 8bbe397caa0242579d9743094319a762 [term 0 FOLLOWER]: Advancing to term 1
I20260809 09:33:02.189816 31994 raft_consensus.cc:2454] T ff306e9576134d12a3f3c5fec97445d8 P 825ed73d552f4f8cbce972a3dc70ccd8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 143f334e51d54d17911a2d2ebb0b6554 in term 1.
I20260809 09:33:02.189975 31793 raft_consensus.cc:2454] T ff306e9576134d12a3f3c5fec97445d8 P 8bbe397caa0242579d9743094319a762 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 143f334e51d54d17911a2d2ebb0b6554 in term 1.
I20260809 09:33:02.190148 31491 leader_election.cc:302] T ff306e9576134d12a3f3c5fec97445d8 P 143f334e51d54d17911a2d2ebb0b6554 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 143f334e51d54d17911a2d2ebb0b6554, 825ed73d552f4f8cbce972a3dc70ccd8; no voters:
I20260809 09:33:02.190382 32242 raft_consensus.cc:2781] T ff306e9576134d12a3f3c5fec97445d8 P 143f334e51d54d17911a2d2ebb0b6554 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 09:33:02.190665 32242 raft_consensus.cc:686] T ff306e9576134d12a3f3c5fec97445d8 P 143f334e51d54d17911a2d2ebb0b6554 [term 1 LEADER]: Becoming Leader. State: Replica: 143f334e51d54d17911a2d2ebb0b6554, State: Running, Role: LEADER
I20260809 09:33:02.190834 32242 consensus_queue.cc:237] T ff306e9576134d12a3f3c5fec97445d8 P 143f334e51d54d17911a2d2ebb0b6554 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bbe397caa0242579d9743094319a762" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 44297 } } peers { permanent_uuid: "825ed73d552f4f8cbce972a3dc70ccd8" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 44279 } } peers { permanent_uuid: "143f334e51d54d17911a2d2ebb0b6554" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 42545 } }
I20260809 09:33:02.191534 32210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:33:02.191268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:37175 (local address 127.20.125.129:51561)
0809 09:33:02.191385 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0809 09:33:02.191449 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.190:37175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260809 09:33:02.192960 31112 catalog_manager.cc:5495] T ff306e9576134d12a3f3c5fec97445d8 P 143f334e51d54d17911a2d2ebb0b6554 reported cstate change: term changed from 0 to 1, leader changed from <none> to 143f334e51d54d17911a2d2ebb0b6554 (127.20.125.129). New cstate: current_term: 1 leader_uuid: "143f334e51d54d17911a2d2ebb0b6554" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bbe397caa0242579d9743094319a762" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 44297 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "825ed73d552f4f8cbce972a3dc70ccd8" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 44279 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "143f334e51d54d17911a2d2ebb0b6554" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 42545 } health_report { overall_health: HEALTHY } } }
I20260809 09:33:02.226097 32233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:33:02.223146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38900 (local address 127.20.125.129:42545)
0809 09:33:02.223310 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0809 09:33:02.223314 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 09:33:02.223329 (+ 15us) server_negotiation.cc:407] Connection header received
0809 09:33:02.223378 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:33:02.223380 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:33:02.223422 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:33:02.223488 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 09:33:02.223751 (+ 263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:33:02.224281 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:33:02.225092 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:33:02.225315 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:33:02.225396 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:33:02.225612 (+ 216us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 09:33:02.225680 (+ 68us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 09:33:02.225734 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0809 09:33:02.225844 (+ 110us) server_negotiation.cc:299] Negotiation successful
0809 09:33:02.225916 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260809 09:33:02.228426 31793 raft_consensus.cc:1261] T ff306e9576134d12a3f3c5fec97445d8 P 8bbe397caa0242579d9743094319a762 [term 1 FOLLOWER]: Refusing update from remote peer 143f334e51d54d17911a2d2ebb0b6554: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 09:33:02.228452 31994 raft_consensus.cc:1261] T ff306e9576134d12a3f3c5fec97445d8 P 825ed73d552f4f8cbce972a3dc70ccd8 [term 1 FOLLOWER]: Refusing update from remote peer 143f334e51d54d17911a2d2ebb0b6554: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 09:33:02.228965 32246 consensus_queue.cc:1035] T ff306e9576134d12a3f3c5fec97445d8 P 143f334e51d54d17911a2d2ebb0b6554 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8bbe397caa0242579d9743094319a762" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 44297 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 09:33:02.229146 32242 consensus_queue.cc:1035] T ff306e9576134d12a3f3c5fec97445d8 P 143f334e51d54d17911a2d2ebb0b6554 [LEADER]: Connected to new peer: Peer: permanent_uuid: "825ed73d552f4f8cbce972a3dc70ccd8" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 44279 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 09:33:02.231180 32207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:33:02.230903 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:37175 (local address 127.20.125.130:37725)
0809 09:33:02.231026 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0809 09:33:02.231097 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.190:37175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260809 09:33:02.231374 32206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:33:02.231058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:37175 (local address 127.20.125.131:48581)
0809 09:33:02.231216 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0809 09:33:02.231280 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.190:37175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
I20260809 09:33:02.234334 32210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:33:02.234104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:37175 (local address 127.20.125.129:45801)
0809 09:33:02.234201 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0809 09:33:02.234255 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.190:37175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260809 09:33:02.236634 32206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:33:02.236319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:37175 (local address 127.20.125.131:60727)
0809 09:33:02.236490 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0809 09:33:02.236553 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.190:37175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90}
I20260809 09:33:02.237162 32210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:33:02.236913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:37175 (local address 127.20.125.129:51771)
0809 09:33:02.237044 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0809 09:33:02.237094 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.190:37175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260809 09:33:02.238955 32207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:33:02.238587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:37175 (local address 127.20.125.130:40661)
0809 09:33:02.238809 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0809 09:33:02.238873 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.190:37175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138}
I20260809 09:33:02.240233 31109 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49646:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 09:33:02.240396 31109 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49646:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 09:33:02.241420 32256 mvcc.cc:205] Tried to move back new op lower bound from 7316553655203135488 to 7316553655056015360. Current Snapshot: MvccSnapshot[applied={T|T < 7316553655203135488 or (T in {7316553655203135488})}]
I20260809 09:33:02.244163 31109 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3c029f42802647dea9a4c2f436be31eb: Sending DeleteTablet for 3 replicas of tablet ff306e9576134d12a3f3c5fec97445d8
I20260809 09:33:02.244952 31552 tablet_service.cc:1514] Processing DeleteTablet for tablet ff306e9576134d12a3f3c5fec97445d8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 09:33:02 UTC) from {username='oryx', principal='oryx/127.20.125.189@KRBTEST.COM'} at 127.0.0.1:38890
I20260809 09:33:02.245047 31765 tablet_service.cc:1514] Processing DeleteTablet for tablet ff306e9576134d12a3f3c5fec97445d8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 09:33:02 UTC) from {username='oryx', principal='oryx/127.20.125.189@KRBTEST.COM'} at 127.0.0.1:53080
I20260809 09:33:02.245316 31960 tablet_service.cc:1514] Processing DeleteTablet for tablet ff306e9576134d12a3f3c5fec97445d8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 09:33:02 UTC) from {username='oryx', principal='oryx/127.20.125.189@KRBTEST.COM'} at 127.0.0.1:39256
I20260809 09:33:02.245390 32265 tablet_replica.cc:330] T ff306e9576134d12a3f3c5fec97445d8 P 8bbe397caa0242579d9743094319a762: stopping tablet replica
I20260809 09:33:02.245532 32265 raft_consensus.cc:2229] T ff306e9576134d12a3f3c5fec97445d8 P 8bbe397caa0242579d9743094319a762 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 09:33:02.245682 32265 raft_consensus.cc:2258] T ff306e9576134d12a3f3c5fec97445d8 P 8bbe397caa0242579d9743094319a762 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 09:33:02.245730 20982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu with pid 31338
I20260809 09:33:02.256417 32265 ts_tablet_manager.cc:1905] T ff306e9576134d12a3f3c5fec97445d8 P 8bbe397caa0242579d9743094319a762: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 09:33:02.258718 32265 ts_tablet_manager.cc:1918] T ff306e9576134d12a3f3c5fec97445d8 P 8bbe397caa0242579d9743094319a762: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 09:33:02.258826 32265 log.cc:1196] T ff306e9576134d12a3f3c5fec97445d8 P 8bbe397caa0242579d9743094319a762: Deleting WAL directory at /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/minicluster-data/ts-1/wal/wals/ff306e9576134d12a3f3c5fec97445d8
I20260809 09:33:02.258983 32266 tablet_replica.cc:330] T ff306e9576134d12a3f3c5fec97445d8 P 825ed73d552f4f8cbce972a3dc70ccd8: stopping tablet replica
I20260809 09:33:02.259114 32265 ts_tablet_manager.cc:1939] T ff306e9576134d12a3f3c5fec97445d8 P 8bbe397caa0242579d9743094319a762: Deleting consensus metadata
I20260809 09:33:02.259218 32266 raft_consensus.cc:2229] T ff306e9576134d12a3f3c5fec97445d8 P 825ed73d552f4f8cbce972a3dc70ccd8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 09:33:02.259406 32266 raft_consensus.cc:2258] T ff306e9576134d12a3f3c5fec97445d8 P 825ed73d552f4f8cbce972a3dc70ccd8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 09:33:02.259963 31089 catalog_manager.cc:4841] TS 8bbe397caa0242579d9743094319a762 (127.20.125.130:44297): tablet ff306e9576134d12a3f3c5fec97445d8 (table test-table [id=1767962b94bf4795a5f6f9bb7cf91ee8]) successfully deleted
I20260809 09:33:02.260313 32266 ts_tablet_manager.cc:1905] T ff306e9576134d12a3f3c5fec97445d8 P 825ed73d552f4f8cbce972a3dc70ccd8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 09:33:02.264003 32266 ts_tablet_manager.cc:1918] T ff306e9576134d12a3f3c5fec97445d8 P 825ed73d552f4f8cbce972a3dc70ccd8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 09:33:02.264114 32266 log.cc:1196] T ff306e9576134d12a3f3c5fec97445d8 P 825ed73d552f4f8cbce972a3dc70ccd8: Deleting WAL directory at /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786267941944493-20982-0/minicluster-data/ts-2/wal/wals/ff306e9576134d12a3f3c5fec97445d8
I20260809 09:33:02.264461 32266 ts_tablet_manager.cc:1939] T ff306e9576134d12a3f3c5fec97445d8 P 825ed73d552f4f8cbce972a3dc70ccd8: Deleting consensus metadata
I20260809 09:33:02.265360 20982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu with pid 31671
I20260809 09:33:02.267740 31087 catalog_manager.cc:4841] TS 825ed73d552f4f8cbce972a3dc70ccd8 (127.20.125.131:44279): tablet ff306e9576134d12a3f3c5fec97445d8 (table test-table [id=1767962b94bf4795a5f6f9bb7cf91ee8]) successfully deleted
I20260809 09:33:02.269856 32232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:33:02.269472 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.129:42545 (local address 127.0.0.1:38904)
0809 09:33:02.269653 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0809 09:33:02.269734 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.129:42545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260809 09:33:02.270251 31088 catalog_manager.cc:4584] TS 143f334e51d54d17911a2d2ebb0b6554 (127.20.125.129:42545): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ff306e9576134d12a3f3c5fec97445d8: Network error: Client connection negotiation failed: client connection to 127.20.125.129:42545: connect: Connection refused (error 111)
I20260809 09:33:02.278007 20982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu with pid 31870
I20260809 09:33:02.291908 20982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu with pid 30901
I20260809 09:33:02.303334 20982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu with pid 31055
2026-08-09T09:33:02Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (7490 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786267941944493-20982-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 09:33:02 dist-test-slave-7kzw krb5kdc[32272](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 09:33:02 dist-test-slave-7kzw krb5kdc[32272](info): set up 2 sockets
Aug 09 09:33:02 dist-test-slave-7kzw krb5kdc[32272](info): commencing operation
krb5kdc: starting...
W20260809 09:33:03.591585 20982 mini_kdc.cc:121] Time spent starting KDC: real 1.260s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786267941944493-20982-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786267941944493-20982-0/krb5kdc/test-user.keytab.
Aug 09 09:33:03 dist-test-slave-7kzw krb5kdc[32272](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786267983, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T09:33:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T09:33:03Z Disabled control of system clock
WARNING: no policy specified for oryx/127.20.125.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.20.125.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.20.125.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786267941944493-20982-0/krb5kdc/oryx_127.20.125.190.keytab.
Entry for principal oryx/127.20.125.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786267941944493-20982-0/krb5kdc/oryx_127.20.125.190.keytab.
I20260809 09:33:03.731463 20982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
/tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786267941944493-20982-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786267941944493-20982-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786267941944493-20982-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786267941944493-20982-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.125.190:43265
--webserver_interface=127.20.125.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786267941944493-20982-0/krb5kdc/oryx_127.20.125.190.keytab
--principal=oryx/127.20.125.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.125.148:34785
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.125.190:43265
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786267941944493-20982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786267941944493-20982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786267941944493-20982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 09:33:03.851985 32291 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 09:33:03.852241 32291 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 09:33:03.852281 32291 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 09:33:03.856271 32291 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 09:33:03.856323 32291 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260809 09:33:03.856340 32291 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 09:33:03.856359 32291 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 09:33:03.856374 32291 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 09:33:03.856444 32291 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 09:33:03.861707 32291 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.125.148:34785
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786267941944493-20982-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786267941944493-20982-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.125.190:43265
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786267941944493-20982-0/krb5kdc/oryx_127.20.125.190.keytab
--principal=oryx/127.20.125.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.125.190:43265
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786267941944493-20982-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.125.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32291
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786267941944493-20982-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 08:38:55 UTC on ccycloud.quasar-njjleu.root.comops.site
build id 4575935
I20260809 09:33:03.862833 32291 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 09:33:03.863909 32291 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 09:33:03.872741 32302 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:33:03.872764 32291 server_base.cc:1034] running on GCE node
W20260809 09:33:03.872790 32305 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:33:03.872789 32301 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:33:03.873636 32291 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 09:33:03.874859 32291 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 09:33:03.876080 32291 hybrid_clock.cc:648] HybridClock initialized: now 1786267983876054 us; error 46 us; skew 500 ppm
Aug 09 09:33:03 dist-test-slave-7kzw krb5kdc[32272](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786267983, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.125.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 09:33:03.879268 32291 init.cc:377] Logged in from keytab as oryx/127.20.125.190@KRBTEST.COM (short username oryx)
I20260809 09:33:03.880256 32291 webserver.cc:460] Webserver started at http://127.20.125.190:34117/ using document root <none> and password file <none>
I20260809 09:33:03.880841 32291 fs_manager.cc:359] Metadata directory not provided
I20260809 09:33:03.880939 32291 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 09:33:03.881184 32291 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 09:33:03.883344 32291 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786267941944493-20982-0/minicluster-data/master-0/data/instance:
uuid: "fdd08fcf074d4c108a4ea936f91b202d"
format_stamp: "Formatted at 2026-08-09 09:33:03 on dist-test-slave-7kzw"
server_key: "8989fa3f5de2abb5466e1d45cdab4367"
server_key_iv: "5cc6ee1357cbd81587c70a5995394f7d"
server_key_version: "encryptionkey@0"
I20260809 09:33:03.883921 32291 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786267941944493-20982-0/minicluster-data/master-0/wal/instance:
uuid: "fdd08fcf074d4c108a4ea936f91b202d"
format_stamp: "Formatted at 2026-08-09 09:33:03 on dist-test-slave-7kzw"
server_key: "8989fa3f5de2abb5466e1d45cdab4367"
server_key_iv: "5cc6ee1357cbd81587c70a5995394f7d"
server_key_version: "encryptionkey@0"
I20260809 09:33:03.887774 32291 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260809 09:33:03.890362 32315 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 09:33:03.891775 32291 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 09:33:03.891944 32291 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786267941944493-20982-0/minicluster-data/master-0/data,/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786267941944493-20982-0/minicluster-data/master-0/wal
uuid: "fdd08fcf074d4c108a4ea936f91b202d"
format_stamp: "Formatted at 2026-08-09 09:33:03 on dist-test-slave-7kzw"
server_key: "8989fa3f5de2abb5466e1d45cdab4367"
server_key_iv: "5cc6ee1357cbd81587c70a5995394f7d"
server_key_version: "encryptionkey@0"
I20260809 09:33:03.892068 32291 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786267941944493-20982-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786267941944493-20982-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786267941944493-20982-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 09:33:03.907454 32291 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 09:33:03.911041 32291 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 09:33:03.911274 32291 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 09:33:03.919775 32291 rpc_server.cc:307] RPC server started. Bound to: 127.20.125.190:43265
I20260809 09:33:03.919868 32402 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.125.190:43265 every 8 connection(s)
I20260809 09:33:03.921136 32291 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786267941944493-20982-0/minicluster-data/master-0/data/info.pb
I20260809 09:33:03.924117 32403 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 09:33:03.929390 20982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu as pid 32291
I20260809 09:33:03.929502 20982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786267941944493-20982-0/minicluster-data/master-0/wal/instance
I20260809 09:33:03.929811 20982 external_mini_cluster.cc:1428] Setting key a3a3d01577c8819f6c44376fe781694d
I20260809 09:33:03.930162 32403 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fdd08fcf074d4c108a4ea936f91b202d: Bootstrap starting.
I20260809 09:33:03.933154 32403 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fdd08fcf074d4c108a4ea936f91b202d: Neither blocks nor log segments found. Creating new log.
I20260809 09:33:03.933977 32403 log.cc:824] T 00000000000000000000000000000000 P fdd08fcf074d4c108a4ea936f91b202d: Log is configured to *not* fsync() on all Append() calls
I20260809 09:33:03.936177 32403 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fdd08fcf074d4c108a4ea936f91b202d: No bootstrap required, opened a new log
Aug 09 09:33:03 dist-test-slave-7kzw krb5kdc[32272](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786267983, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.20.125.190@KRBTEST.COM
I20260809 09:33:03.941025 32403 raft_consensus.cc:348] T 00000000000000000000000000000000 P fdd08fcf074d4c108a4ea936f91b202d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fdd08fcf074d4c108a4ea936f91b202d" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 43265 } }
I20260809 09:33:03.941370 32403 raft_consensus.cc:374] T 00000000000000000000000000000000 P fdd08fcf074d4c108a4ea936f91b202d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 09:33:03.941439 32403 raft_consensus.cc:729] T 00000000000000000000000000000000 P fdd08fcf074d4c108a4ea936f91b202d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fdd08fcf074d4c108a4ea936f91b202d, State: Initialized, Role: FOLLOWER
I20260809 09:33:03.941977 32403 consensus_queue.cc:260] T 00000000000000000000000000000000 P fdd08fcf074d4c108a4ea936f91b202d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fdd08fcf074d4c108a4ea936f91b202d" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 43265 } }
I20260809 09:33:03.942166 32403 raft_consensus.cc:388] T 00000000000000000000000000000000 P fdd08fcf074d4c108a4ea936f91b202d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 09:33:03.942241 32403 raft_consensus.cc:482] T 00000000000000000000000000000000 P fdd08fcf074d4c108a4ea936f91b202d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 09:33:03.942339 32403 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fdd08fcf074d4c108a4ea936f91b202d [term 0 FOLLOWER]: Advancing to term 1
I20260809 09:33:03.943593 32403 raft_consensus.cc:504] T 00000000000000000000000000000000 P fdd08fcf074d4c108a4ea936f91b202d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fdd08fcf074d4c108a4ea936f91b202d" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 43265 } }
I20260809 09:33:03.944003 32403 leader_election.cc:302] T 00000000000000000000000000000000 P fdd08fcf074d4c108a4ea936f91b202d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fdd08fcf074d4c108a4ea936f91b202d; no voters:
I20260809 09:33:03.944401 32403 leader_election.cc:288] T 00000000000000000000000000000000 P fdd08fcf074d4c108a4ea936f91b202d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 09:33:03.944473 32411 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fdd08fcf074d4c108a4ea936f91b202d [term 1 FOLLOWER]: Leader election won for term 1
I20260809 09:33:03.944762 32411 raft_consensus.cc:686] T 00000000000000000000000000000000 P fdd08fcf074d4c108a4ea936f91b202d [term 1 LEADER]: Becoming Leader. State: Replica: fdd08fcf074d4c108a4ea936f91b202d, State: Running, Role: LEADER
I20260809 09:33:03.945075 32411 consensus_queue.cc:237] T 00000000000000000000000000000000 P fdd08fcf074d4c108a4ea936f91b202d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fdd08fcf074d4c108a4ea936f91b202d" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 43265 } }
I20260809 09:33:03.945818 32403 sys_catalog.cc:564] T 00000000000000000000000000000000 P fdd08fcf074d4c108a4ea936f91b202d [sys.catalog]: configured and running, proceeding with master startup.
I20260809 09:33:03.946830 32415 sys_catalog.cc:455] T 00000000000000000000000000000000 P fdd08fcf074d4c108a4ea936f91b202d [sys.catalog]: SysCatalogTable state changed. Reason: New leader fdd08fcf074d4c108a4ea936f91b202d. Latest consensus state: current_term: 1 leader_uuid: "fdd08fcf074d4c108a4ea936f91b202d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fdd08fcf074d4c108a4ea936f91b202d" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 43265 } } }
I20260809 09:33:03.946959 32415 sys_catalog.cc:458] T 00000000000000000000000000000000 P fdd08fcf074d4c108a4ea936f91b202d [sys.catalog]: This master's current role is: LEADER
I20260809 09:33:03.947250 32413 sys_catalog.cc:455] T 00000000000000000000000000000000 P fdd08fcf074d4c108a4ea936f91b202d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fdd08fcf074d4c108a4ea936f91b202d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fdd08fcf074d4c108a4ea936f91b202d" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 43265 } } }
I20260809 09:33:03.947314 32409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:33:03.931355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39898 (local address 127.20.125.190:43265)
0809 09:33:03.932686 (+ 1331us) server_negotiation.cc:206] Beginning negotiation
0809 09:33:03.932713 (+ 27us) server_negotiation.cc:399] Waiting for connection header
0809 09:33:03.932904 (+ 191us) server_negotiation.cc:407] Connection header received
0809 09:33:03.933754 (+ 850us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:33:03.933773 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:33:03.934153 (+ 380us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:33:03.934538 (+ 385us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:33:03.935432 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:33:03.936489 (+ 1057us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:33:03.937673 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:33:03.938146 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:33:03.938274 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:33:03.941220 (+ 2946us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:33:03.941226 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:33:03.941243 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:33:03.941276 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:33:03.943407 (+ 2131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:33:03.943898 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:33:03.943904 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:33:03.943910 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:33:03.944058 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:33:03.944390 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:33:03.944394 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:33:03.944395 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:33:03.944958 (+ 563us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:33:03.945732 (+ 774us) server_negotiation.cc:1035] Waiting for connection context
0809 09:33:03.945888 (+ 156us) server_negotiation.cc:299] Negotiation successful
0809 09:33:03.946198 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1165,"thread_start_us":986,"threads_started":1}
I20260809 09:33:03.947364 32413 sys_catalog.cc:458] T 00000000000000000000000000000000 P fdd08fcf074d4c108a4ea936f91b202d [sys.catalog]: This master's current role is: LEADER
W20260809 09:33:03.950804 32436 catalog_manager.cc:1558] T 00000000000000000000000000000000 P fdd08fcf074d4c108a4ea936f91b202d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 09:33:03.950887 32436 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 09:33:03.950944 32425 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 09:33:03.951822 32425 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 09:33:03.957958 32425 catalog_manager.cc:1347] Generated new cluster ID: 0716907ec5224b64bbc4e8be25751cd6
I20260809 09:33:03.958042 32425 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 09:33:03.986244 32425 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 09:33:03.987322 32425 catalog_manager.cc:1504] Loading token signing keys...
I20260809 09:33:04.000839 32425 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fdd08fcf074d4c108a4ea936f91b202d: Generated new TSK 0
I20260809 09:33:04.001973 32425 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.20.125.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.20.125.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.20.125.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786267941944493-20982-0/krb5kdc/oryx_127.20.125.129.keytab.
Entry for principal oryx/127.20.125.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786267941944493-20982-0/krb5kdc/oryx_127.20.125.129.keytab.
I20260809 09:33:04.042023 20982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
/tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786267941944493-20982-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786267941944493-20982-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786267941944493-20982-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786267941944493-20982-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.125.129:0
--local_ip_for_outbound_sockets=127.20.125.129
--webserver_interface=127.20.125.129
--webserver_port=0
--tserver_master_addrs=127.20.125.190:43265
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786267941944493-20982-0/krb5kdc/oryx_127.20.125.129.keytab
--principal=oryx/127.20.125.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.125.148:34785
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786267941944493-20982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786267941944493-20982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786267941944493-20982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 09:33:04.160971 32446 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 09:33:04.161227 32446 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 09:33:04.161271 32446 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 09:33:04.165220 32446 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 09:33:04.165270 32446 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 09:33:04.165307 32446 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 09:33:04.165383 32446 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.125.129
I20260809 09:33:04.170496 32446 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.125.148:34785
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786267941944493-20982-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786267941944493-20982-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786267941944493-20982-0/krb5kdc/oryx_127.20.125.129.keytab
--principal=oryx/127.20.125.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.125.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786267941944493-20982-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.125.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.20.125.190:43265
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32446
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.125.129
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786267941944493-20982-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 08:38:55 UTC on ccycloud.quasar-njjleu.root.comops.site
build id 4575935
I20260809 09:33:04.171532 32446 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 09:33:04.172569 32446 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 09:33:04.182720 32458 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:33:04.182730 32455 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:33:04.182731 32456 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:33:04.183180 32446 server_base.cc:1034] running on GCE node
I20260809 09:33:04.183578 32446 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 09:33:04.184180 32446 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 09:33:04.185356 32446 hybrid_clock.cc:648] HybridClock initialized: now 1786267984185337 us; error 32 us; skew 500 ppm
Aug 09 09:33:04 dist-test-slave-7kzw krb5kdc[32272](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786267984, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.125.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 09:33:04.188174 32446 init.cc:377] Logged in from keytab as oryx/127.20.125.129@KRBTEST.COM (short username oryx)
I20260809 09:33:04.189065 32446 webserver.cc:460] Webserver started at http://127.20.125.129:34005/ using document root <none> and password file <none>
I20260809 09:33:04.189654 32446 fs_manager.cc:359] Metadata directory not provided
I20260809 09:33:04.189770 32446 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 09:33:04.190002 32446 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 09:33:04.192104 32446 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786267941944493-20982-0/minicluster-data/ts-0/data/instance:
uuid: "2a843b4644cd4b2389cf749e5c3f2e58"
format_stamp: "Formatted at 2026-08-09 09:33:04 on dist-test-slave-7kzw"
server_key: "442d8c96e98f5781244851f04833c0f5"
server_key_iv: "5f0efffb5180f4b77b63bf68b4bff53d"
server_key_version: "encryptionkey@0"
I20260809 09:33:04.192689 32446 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786267941944493-20982-0/minicluster-data/ts-0/wal/instance:
uuid: "2a843b4644cd4b2389cf749e5c3f2e58"
format_stamp: "Formatted at 2026-08-09 09:33:04 on dist-test-slave-7kzw"
server_key: "442d8c96e98f5781244851f04833c0f5"
server_key_iv: "5f0efffb5180f4b77b63bf68b4bff53d"
server_key_version: "encryptionkey@0"
I20260809 09:33:04.196856 32446 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.002s
I20260809 09:33:04.199493 32467 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 09:33:04.200894 32446 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260809 09:33:04.201041 32446 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786267941944493-20982-0/minicluster-data/ts-0/data,/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786267941944493-20982-0/minicluster-data/ts-0/wal
uuid: "2a843b4644cd4b2389cf749e5c3f2e58"
format_stamp: "Formatted at 2026-08-09 09:33:04 on dist-test-slave-7kzw"
server_key: "442d8c96e98f5781244851f04833c0f5"
server_key_iv: "5f0efffb5180f4b77b63bf68b4bff53d"
server_key_version: "encryptionkey@0"
I20260809 09:33:04.201166 32446 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786267941944493-20982-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786267941944493-20982-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786267941944493-20982-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 09:33:04.212927 32446 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 09:33:04.216320 32446 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 09:33:04.216550 32446 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 09:33:04.218093 32446 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 09:33:04.218176 32446 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 09:33:04.218251 32446 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 09:33:04.218304 32446 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 09:33:04 dist-test-slave-7kzw krb5kdc[32272](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786267984, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.125.129@KRBTEST.COM for oryx/127.20.125.190@KRBTEST.COM
I20260809 09:33:04.230245 32446 rpc_server.cc:307] RPC server started. Bound to: 127.20.125.129:40235
I20260809 09:33:04.230631 32646 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.125.129:40235 every 8 connection(s)
I20260809 09:33:04.231262 32446 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786267941944493-20982-0/minicluster-data/ts-0/data/info.pb
I20260809 09:33:04.233078 32409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:33:04.219186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.129:38347 (local address 127.20.125.190:43265)
0809 09:33:04.219344 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0809 09:33:04.219349 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 09:33:04.220490 (+ 1141us) server_negotiation.cc:407] Connection header received
0809 09:33:04.221870 (+ 1380us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:33:04.221873 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:33:04.221937 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:33:04.222036 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:33:04.223650 (+ 1614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:33:04.224248 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:33:04.225488 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:33:04.225792 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:33:04.225939 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:33:04.229011 (+ 3072us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:33:04.229014 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:33:04.229016 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:33:04.229042 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:33:04.230834 (+ 1792us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:33:04.231422 (+ 588us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:33:04.231425 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:33:04.231427 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:33:04.231518 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:33:04.231940 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:33:04.231943 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:33:04.231944 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:33:04.232159 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:33:04.232275 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0809 09:33:04.232761 (+ 486us) server_negotiation.cc:299] Negotiation successful
0809 09:33:04.232905 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260809 09:33:04.233806 32483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:33:04.219492 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:43265 (local address 127.20.125.129:38347)
0809 09:33:04.220330 (+ 838us) negotiation.cc:107] Waiting for socket to connect
0809 09:33:04.220376 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 09:33:04.221600 (+ 1224us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:33:04.222347 (+ 747us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:33:04.222354 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:33:04.222856 (+ 502us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:33:04.223484 (+ 628us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:33:04.223498 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:33:04.224388 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:33:04.224392 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:33:04.225345 (+ 953us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:33:04.225354 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:33:04.226172 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:33:04.226177 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:33:04.226436 (+ 259us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:33:04.227142 (+ 706us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 09:33:04.227160 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:33:04.228820 (+ 1660us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 09:33:04.230952 (+ 2132us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:33:04.230958 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:33:04.230972 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:33:04.231328 (+ 356us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:33:04.231614 (+ 286us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:33:04.231618 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:33:04.231620 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:33:04.231821 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:33:04.232273 (+ 452us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 09:33:04.232280 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 09:33:04.232520 (+ 240us) client_negotiation.cc:769] Sending connection context
0809 09:33:04.232708 (+ 188us) client_negotiation.cc:240] Negotiation successful
0809 09:33:04.233019 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":623,"thread_start_us":133,"threads_started":1}
I20260809 09:33:04.235260 32647 heartbeater.cc:347] Connected to a master server at 127.20.125.190:43265
I20260809 09:33:04.235430 32647 heartbeater.cc:464] Registering TS with master...
I20260809 09:33:04.235929 32647 heartbeater.cc:511] Master 127.20.125.190:43265 requested a full tablet report, sending...
I20260809 09:33:04.237702 32336 ts_manager.cc:194] Registered new tserver with Master: 2a843b4644cd4b2389cf749e5c3f2e58 (127.20.125.129:40235)
I20260809 09:33:04.239374 32336 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.20.125.129@KRBTEST.COM'} at 127.20.125.129:38347
I20260809 09:33:04.240525 20982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu as pid 32446
I20260809 09:33:04.240661 20982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786267941944493-20982-0/minicluster-data/ts-0/wal/instance
I20260809 09:33:04.241020 20982 external_mini_cluster.cc:1428] Setting key 6e07a6bcc3a57dab0e627bda6219eadf
I20260809 09:33:04.245718 32654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:33:04.237620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:43265 (local address 127.20.125.129:44847)
0809 09:33:04.237998 (+ 378us) negotiation.cc:107] Waiting for socket to connect
0809 09:33:04.238011 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 09:33:04.238094 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:33:04.238332 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:33:04.238334 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:33:04.238573 (+ 239us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:33:04.238672 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:33:04.238678 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:33:04.239724 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:33:04.239727 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:33:04.240430 (+ 703us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:33:04.240441 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:33:04.241130 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:33:04.241137 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:33:04.241236 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:33:04.241820 (+ 584us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 09:33:04.241835 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:33:04.242280 (+ 445us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 09:33:04.244106 (+ 1826us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:33:04.244112 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:33:04.244117 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:33:04.244397 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:33:04.244717 (+ 320us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:33:04.244723 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:33:04.244726 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:33:04.244842 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:33:04.245288 (+ 446us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 09:33:04.245290 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 09:33:04.245364 (+ 74us) client_negotiation.cc:769] Sending connection context
0809 09:33:04.245443 (+ 79us) client_negotiation.cc:240] Negotiation successful
0809 09:33:04.245543 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":308,"thread_start_us":143,"threads_started":1}
I20260809 09:33:04.246047 32409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:33:04.237731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.129:44847 (local address 127.20.125.190:43265)
0809 09:33:04.237854 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0809 09:33:04.237859 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 09:33:04.238092 (+ 233us) server_negotiation.cc:407] Connection header received
0809 09:33:04.238178 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:33:04.238181 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:33:04.238225 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:33:04.238307 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:33:04.238789 (+ 482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:33:04.239583 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:33:04.240605 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:33:04.240941 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:33:04.241063 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:33:04.242450 (+ 1387us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:33:04.242456 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:33:04.242460 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:33:04.242508 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:33:04.243987 (+ 1479us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:33:04.244502 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:33:04.244504 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:33:04.244505 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:33:04.244581 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:33:04.244943 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:33:04.244947 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:33:04.244949 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:33:04.245190 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:33:04.245297 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0809 09:33:04.245529 (+ 232us) server_negotiation.cc:299] Negotiation successful
0809 09:33:04.245660 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
WARNING: no policy specified for oryx/127.20.125.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.20.125.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.20.125.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786267941944493-20982-0/krb5kdc/oryx_127.20.125.130.keytab.
Entry for principal oryx/127.20.125.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786267941944493-20982-0/krb5kdc/oryx_127.20.125.130.keytab.
I20260809 09:33:04.269922 20982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
/tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786267941944493-20982-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786267941944493-20982-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786267941944493-20982-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786267941944493-20982-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.125.130:0
--local_ip_for_outbound_sockets=127.20.125.130
--webserver_interface=127.20.125.130
--webserver_port=0
--tserver_master_addrs=127.20.125.190:43265
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786267941944493-20982-0/krb5kdc/oryx_127.20.125.130.keytab
--principal=oryx/127.20.125.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.125.148:34785
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786267941944493-20982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786267941944493-20982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786267941944493-20982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 09:33:04.382014 32658 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 09:33:04.382256 32658 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 09:33:04.382300 32658 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 09:33:04.386350 32658 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 09:33:04.386406 32658 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 09:33:04.386509 32658 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 09:33:04.386569 32658 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.125.130
I20260809 09:33:04.391855 32658 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.125.148:34785
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786267941944493-20982-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786267941944493-20982-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786267941944493-20982-0/krb5kdc/oryx_127.20.125.130.keytab
--principal=oryx/127.20.125.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.125.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786267941944493-20982-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.125.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.20.125.190:43265
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32658
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.125.130
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786267941944493-20982-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 08:38:55 UTC on ccycloud.quasar-njjleu.root.comops.site
build id 4575935
I20260809 09:33:04.392848 32658 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 09:33:04.393857 32658 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 09:33:04.403527 32667 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:33:04.403522 32670 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:33:04.403525 32665 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:33:04.403899 32658 server_base.cc:1034] running on GCE node
I20260809 09:33:04.404306 32658 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 09:33:04.404920 32658 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 09:33:04.406108 32658 hybrid_clock.cc:648] HybridClock initialized: now 1786267984406087 us; error 31 us; skew 500 ppm
Aug 09 09:33:04 dist-test-slave-7kzw krb5kdc[32272](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786267984, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.125.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 09:33:04.409075 32658 init.cc:377] Logged in from keytab as oryx/127.20.125.130@KRBTEST.COM (short username oryx)
I20260809 09:33:04.410005 32658 webserver.cc:460] Webserver started at http://127.20.125.130:44951/ using document root <none> and password file <none>
I20260809 09:33:04.410632 32658 fs_manager.cc:359] Metadata directory not provided
I20260809 09:33:04.410737 32658 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 09:33:04.410962 32658 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 09:33:04.412972 32658 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786267941944493-20982-0/minicluster-data/ts-1/data/instance:
uuid: "b44b10e0ca7c4c9e89325813a36288d7"
format_stamp: "Formatted at 2026-08-09 09:33:04 on dist-test-slave-7kzw"
server_key: "8ab6d2ed7686ef4efb791daae76bbb7f"
server_key_iv: "4b1776f58522fb94d51f1c3398015f06"
server_key_version: "encryptionkey@0"
I20260809 09:33:04.413529 32658 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786267941944493-20982-0/minicluster-data/ts-1/wal/instance:
uuid: "b44b10e0ca7c4c9e89325813a36288d7"
format_stamp: "Formatted at 2026-08-09 09:33:04 on dist-test-slave-7kzw"
server_key: "8ab6d2ed7686ef4efb791daae76bbb7f"
server_key_iv: "4b1776f58522fb94d51f1c3398015f06"
server_key_version: "encryptionkey@0"
I20260809 09:33:04.417265 32658 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260809 09:33:04.419710 32684 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 09:33:04.420964 32658 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 09:33:04.421085 32658 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786267941944493-20982-0/minicluster-data/ts-1/data,/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786267941944493-20982-0/minicluster-data/ts-1/wal
uuid: "b44b10e0ca7c4c9e89325813a36288d7"
format_stamp: "Formatted at 2026-08-09 09:33:04 on dist-test-slave-7kzw"
server_key: "8ab6d2ed7686ef4efb791daae76bbb7f"
server_key_iv: "4b1776f58522fb94d51f1c3398015f06"
server_key_version: "encryptionkey@0"
I20260809 09:33:04.421195 32658 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786267941944493-20982-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786267941944493-20982-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786267941944493-20982-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 09:33:04.435652 32658 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 09:33:04.438689 32658 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 09:33:04.438886 32658 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 09:33:04.440367 32658 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 09:33:04.440442 32658 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 09:33:04.440517 32658 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 09:33:04.440567 32658 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 09:33:04 dist-test-slave-7kzw krb5kdc[32272](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786267984, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.125.130@KRBTEST.COM for oryx/127.20.125.190@KRBTEST.COM
I20260809 09:33:04.451927 32658 rpc_server.cc:307] RPC server started. Bound to: 127.20.125.130:33845
I20260809 09:33:04.452060 386 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.125.130:33845 every 8 connection(s)
I20260809 09:33:04.453024 32658 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786267941944493-20982-0/minicluster-data/ts-1/data/info.pb
I20260809 09:33:04.454268 32409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:33:04.441162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.130:48351 (local address 127.20.125.190:43265)
0809 09:33:04.441299 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0809 09:33:04.441303 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 09:33:04.442332 (+ 1029us) server_negotiation.cc:407] Connection header received
0809 09:33:04.443332 (+ 1000us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:33:04.443338 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:33:04.443407 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:33:04.443508 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:33:04.444669 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:33:04.445580 (+ 911us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:33:04.447352 (+ 1772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:33:04.447609 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:33:04.447700 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:33:04.450402 (+ 2702us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:33:04.450405 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:33:04.450407 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:33:04.450430 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:33:04.452057 (+ 1627us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:33:04.452632 (+ 575us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:33:04.452635 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:33:04.452636 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:33:04.452735 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:33:04.453130 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:33:04.453133 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:33:04.453135 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:33:04.453320 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:33:04.453433 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0809 09:33:04.453973 (+ 540us) server_negotiation.cc:299] Negotiation successful
0809 09:33:04.454094 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260809 09:33:04.455123 32698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:33:04.441711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:43265 (local address 127.20.125.130:48351)
0809 09:33:04.442170 (+ 459us) negotiation.cc:107] Waiting for socket to connect
0809 09:33:04.442225 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0809 09:33:04.443085 (+ 860us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:33:04.443784 (+ 699us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:33:04.443791 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:33:04.444099 (+ 308us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:33:04.444543 (+ 444us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:33:04.444555 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:33:04.445768 (+ 1213us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:33:04.445772 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:33:04.447156 (+ 1384us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:33:04.447167 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:33:04.447763 (+ 596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:33:04.447770 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:33:04.448013 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:33:04.448700 (+ 687us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 09:33:04.448719 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:33:04.450192 (+ 1473us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 09:33:04.452184 (+ 1992us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:33:04.452190 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:33:04.452202 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:33:04.452519 (+ 317us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:33:04.452830 (+ 311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:33:04.452833 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:33:04.452835 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:33:04.453017 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:33:04.453433 (+ 416us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 09:33:04.453440 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 09:33:04.453732 (+ 292us) client_negotiation.cc:769] Sending connection context
0809 09:33:04.453928 (+ 196us) client_negotiation.cc:240] Negotiation successful
0809 09:33:04.454235 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":276,"thread_start_us":124,"threads_started":1}
I20260809 09:33:04.456240 387 heartbeater.cc:347] Connected to a master server at 127.20.125.190:43265
I20260809 09:33:04.456403 387 heartbeater.cc:464] Registering TS with master...
I20260809 09:33:04.456869 387 heartbeater.cc:511] Master 127.20.125.190:43265 requested a full tablet report, sending...
I20260809 09:33:04.457224 20982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu as pid 32658
I20260809 09:33:04.457341 20982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786267941944493-20982-0/minicluster-data/ts-1/wal/instance
I20260809 09:33:04.457682 20982 external_mini_cluster.cc:1428] Setting key a09cf8c75cacc564d1533780cd419155
I20260809 09:33:04.457927 32340 ts_manager.cc:194] Registered new tserver with Master: b44b10e0ca7c4c9e89325813a36288d7 (127.20.125.130:33845)
I20260809 09:33:04.458645 32340 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.20.125.130@KRBTEST.COM'} at 127.20.125.130:48351
I20260809 09:33:04.466840 32409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:33:04.459236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.130:48053 (local address 127.20.125.190:43265)
0809 09:33:04.459350 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0809 09:33:04.459355 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 09:33:04.459549 (+ 194us) server_negotiation.cc:407] Connection header received
0809 09:33:04.459643 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:33:04.459649 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:33:04.459703 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:33:04.459782 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:33:04.460384 (+ 602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:33:04.461096 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:33:04.461992 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:33:04.462357 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:33:04.462498 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:33:04.463625 (+ 1127us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:33:04.463628 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:33:04.463630 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:33:04.463651 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:33:04.465055 (+ 1404us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:33:04.465568 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:33:04.465570 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:33:04.465572 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:33:04.465645 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:33:04.465963 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:33:04.465965 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:33:04.465966 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:33:04.466153 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:33:04.466242 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0809 09:33:04.466640 (+ 398us) server_negotiation.cc:299] Negotiation successful
0809 09:33:04.466733 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260809 09:33:04.466909 396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:33:04.459114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:43265 (local address 127.20.125.130:48053)
0809 09:33:04.459444 (+ 330us) negotiation.cc:107] Waiting for socket to connect
0809 09:33:04.459461 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 09:33:04.459565 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:33:04.459821 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:33:04.459826 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:33:04.460059 (+ 233us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:33:04.460244 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:33:04.460257 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:33:04.461240 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:33:04.461246 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:33:04.461871 (+ 625us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:33:04.461880 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:33:04.462520 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:33:04.462524 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:33:04.462600 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:33:04.463025 (+ 425us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 09:33:04.463044 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:33:04.463488 (+ 444us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 09:33:04.465170 (+ 1682us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:33:04.465181 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:33:04.465185 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:33:04.465468 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:33:04.465741 (+ 273us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:33:04.465748 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:33:04.465752 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:33:04.465862 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:33:04.466253 (+ 391us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 09:33:04.466258 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 09:33:04.466384 (+ 126us) client_negotiation.cc:769] Sending connection context
0809 09:33:04.466532 (+ 148us) client_negotiation.cc:240] Negotiation successful
0809 09:33:04.466681 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":237,"mutex_wait_us":1,"thread_start_us":115,"threads_started":1}
WARNING: no policy specified for oryx/127.20.125.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.20.125.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.20.125.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786267941944493-20982-0/krb5kdc/oryx_127.20.125.131.keytab.
Entry for principal oryx/127.20.125.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786267941944493-20982-0/krb5kdc/oryx_127.20.125.131.keytab.
I20260809 09:33:04.489429 20982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
/tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786267941944493-20982-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786267941944493-20982-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786267941944493-20982-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786267941944493-20982-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.125.131:0
--local_ip_for_outbound_sockets=127.20.125.131
--webserver_interface=127.20.125.131
--webserver_port=0
--tserver_master_addrs=127.20.125.190:43265
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786267941944493-20982-0/krb5kdc/oryx_127.20.125.131.keytab
--principal=oryx/127.20.125.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.125.148:34785
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786267941944493-20982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786267941944493-20982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786267941944493-20982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 09:33:04.604398 399 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 09:33:04.604713 399 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 09:33:04.604805 399 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 09:33:04.608970 399 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 09:33:04.609031 399 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 09:33:04.609071 399 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 09:33:04.609103 399 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.125.131
I20260809 09:33:04.614228 399 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.125.148:34785
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786267941944493-20982-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786267941944493-20982-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786267941944493-20982-0/krb5kdc/oryx_127.20.125.131.keytab
--principal=oryx/127.20.125.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.125.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786267941944493-20982-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.125.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.20.125.190:43265
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.399
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.125.131
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786267941944493-20982-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 08:38:55 UTC on ccycloud.quasar-njjleu.root.comops.site
build id 4575935
I20260809 09:33:04.615281 399 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 09:33:04.616334 399 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 09:33:04.626044 405 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:33:04.626026 406 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:33:04.626026 409 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:33:04.626535 399 server_base.cc:1034] running on GCE node
I20260809 09:33:04.626978 399 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 09:33:04.627617 399 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 09:33:04.628816 399 hybrid_clock.cc:648] HybridClock initialized: now 1786267984628801 us; error 25 us; skew 500 ppm
Aug 09 09:33:04 dist-test-slave-7kzw krb5kdc[32272](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786267984, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.125.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 09:33:04.631678 399 init.cc:377] Logged in from keytab as oryx/127.20.125.131@KRBTEST.COM (short username oryx)
I20260809 09:33:04.632522 399 webserver.cc:460] Webserver started at http://127.20.125.131:34877/ using document root <none> and password file <none>
I20260809 09:33:04.633083 399 fs_manager.cc:359] Metadata directory not provided
I20260809 09:33:04.633201 399 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 09:33:04.633436 399 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 09:33:04.635509 399 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786267941944493-20982-0/minicluster-data/ts-2/data/instance:
uuid: "2313d46fd7f84023b99b769f25a2bd37"
format_stamp: "Formatted at 2026-08-09 09:33:04 on dist-test-slave-7kzw"
server_key: "480b33849a54d31afcc855f349b2be8d"
server_key_iv: "69e5aacc445b616f26dedf349b2306ee"
server_key_version: "encryptionkey@0"
I20260809 09:33:04.636193 399 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786267941944493-20982-0/minicluster-data/ts-2/wal/instance:
uuid: "2313d46fd7f84023b99b769f25a2bd37"
format_stamp: "Formatted at 2026-08-09 09:33:04 on dist-test-slave-7kzw"
server_key: "480b33849a54d31afcc855f349b2be8d"
server_key_iv: "69e5aacc445b616f26dedf349b2306ee"
server_key_version: "encryptionkey@0"
I20260809 09:33:04.639943 399 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260809 09:33:04.642371 419 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 09:33:04.643610 399 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 09:33:04.643718 399 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786267941944493-20982-0/minicluster-data/ts-2/data,/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786267941944493-20982-0/minicluster-data/ts-2/wal
uuid: "2313d46fd7f84023b99b769f25a2bd37"
format_stamp: "Formatted at 2026-08-09 09:33:04 on dist-test-slave-7kzw"
server_key: "480b33849a54d31afcc855f349b2be8d"
server_key_iv: "69e5aacc445b616f26dedf349b2306ee"
server_key_version: "encryptionkey@0"
I20260809 09:33:04.643850 399 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786267941944493-20982-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786267941944493-20982-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786267941944493-20982-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 09:33:04.668998 399 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 09:33:04.672324 399 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 09:33:04.672529 399 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 09:33:04.674041 399 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 09:33:04.674091 399 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 09:33:04.674156 399 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 09:33:04.674188 399 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 09:33:04 dist-test-slave-7kzw krb5kdc[32272](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786267984, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.125.131@KRBTEST.COM for oryx/127.20.125.190@KRBTEST.COM
I20260809 09:33:04.685322 399 rpc_server.cc:307] RPC server started. Bound to: 127.20.125.131:40493
I20260809 09:33:04.685441 583 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.125.131:40493 every 8 connection(s)
I20260809 09:33:04.686414 399 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786267941944493-20982-0/minicluster-data/ts-2/data/info.pb
I20260809 09:33:04.686802 20982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu as pid 399
I20260809 09:33:04.686949 20982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786267941944493-20982-0/minicluster-data/ts-2/wal/instance
I20260809 09:33:04.687322 20982 external_mini_cluster.cc:1428] Setting key 622119aeb07ef930d6e27fd9639894a7
I20260809 09:33:04.689491 436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:33:04.675581 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:43265 (local address 127.20.125.131:48271)
0809 09:33:04.676090 (+ 509us) negotiation.cc:107] Waiting for socket to connect
0809 09:33:04.676133 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 09:33:04.677137 (+ 1004us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:33:04.677764 (+ 627us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:33:04.677790 (+ 26us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:33:04.678149 (+ 359us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:33:04.678670 (+ 521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:33:04.678686 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:33:04.679566 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:33:04.679570 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:33:04.680677 (+ 1107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:33:04.680687 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:33:04.681212 (+ 525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:33:04.681217 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:33:04.681425 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:33:04.682043 (+ 618us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 09:33:04.682062 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:33:04.684211 (+ 2149us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 09:33:04.686277 (+ 2066us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:33:04.686282 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:33:04.686292 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:33:04.686653 (+ 361us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:33:04.686999 (+ 346us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:33:04.687003 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:33:04.687005 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:33:04.687189 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:33:04.687706 (+ 517us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 09:33:04.687712 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 09:33:04.688002 (+ 290us) client_negotiation.cc:769] Sending connection context
0809 09:33:04.688203 (+ 201us) client_negotiation.cc:240] Negotiation successful
0809 09:33:04.688517 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":311,"thread_start_us":145,"threads_started":1}
I20260809 09:33:04.690106 32409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:33:04.675023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.131:48271 (local address 127.20.125.190:43265)
0809 09:33:04.675152 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0809 09:33:04.675155 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 09:33:04.676244 (+ 1089us) server_negotiation.cc:407] Connection header received
0809 09:33:04.677387 (+ 1143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:33:04.677391 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:33:04.677461 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:33:04.677568 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:33:04.678823 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:33:04.679438 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:33:04.680808 (+ 1370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:33:04.681038 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:33:04.681152 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:33:04.684377 (+ 3225us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:33:04.684380 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:33:04.684383 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:33:04.684416 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:33:04.686142 (+ 1726us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:33:04.686784 (+ 642us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:33:04.686789 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:33:04.686791 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:33:04.686899 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:33:04.687309 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:33:04.687312 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:33:04.687314 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:33:04.687549 (+ 235us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:33:04.687649 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0809 09:33:04.689903 (+ 2254us) server_negotiation.cc:299] Negotiation successful
0809 09:33:04.690001 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260809 09:33:04.690948 585 heartbeater.cc:347] Connected to a master server at 127.20.125.190:43265
I20260809 09:33:04.691107 585 heartbeater.cc:464] Registering TS with master...
I20260809 09:33:04.691602 585 heartbeater.cc:511] Master 127.20.125.190:43265 requested a full tablet report, sending...
I20260809 09:33:04.692590 32336 ts_manager.cc:194] Registered new tserver with Master: 2313d46fd7f84023b99b769f25a2bd37 (127.20.125.131:40493)
I20260809 09:33:04.693231 32336 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.20.125.131@KRBTEST.COM'} at 127.20.125.131:48271
I20260809 09:33:04.700114 596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:33:04.693108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:43265 (local address 127.20.125.131:42393)
0809 09:33:04.693449 (+ 341us) negotiation.cc:107] Waiting for socket to connect
0809 09:33:04.693466 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 09:33:04.693565 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:33:04.693813 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:33:04.693816 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:33:04.694001 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:33:04.694098 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:33:04.694104 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:33:04.694949 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:33:04.694952 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:33:04.695605 (+ 653us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:33:04.695614 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:33:04.696072 (+ 458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:33:04.696076 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:33:04.696173 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:33:04.696696 (+ 523us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 09:33:04.696708 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:33:04.697017 (+ 309us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 09:33:04.698563 (+ 1546us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:33:04.698567 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:33:04.698570 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:33:04.698779 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:33:04.699076 (+ 297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:33:04.699080 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:33:04.699082 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:33:04.699169 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:33:04.699542 (+ 373us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 09:33:04.699546 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 09:33:04.699663 (+ 117us) client_negotiation.cc:769] Sending connection context
0809 09:33:04.699775 (+ 112us) client_negotiation.cc:240] Negotiation successful
0809 09:33:04.699914 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":256,"thread_start_us":110,"threads_started":1}
I20260809 09:33:04.700104 32409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:33:04.693167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.131:42393 (local address 127.20.125.190:43265)
0809 09:33:04.693298 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0809 09:33:04.693303 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 09:33:04.693545 (+ 242us) server_negotiation.cc:407] Connection header received
0809 09:33:04.693660 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:33:04.693662 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:33:04.693706 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:33:04.693795 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:33:04.694265 (+ 470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:33:04.694826 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:33:04.695741 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:33:04.695960 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:33:04.696068 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:33:04.697156 (+ 1088us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:33:04.697160 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:33:04.697162 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:33:04.697186 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:33:04.698433 (+ 1247us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:33:04.698896 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:33:04.698899 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:33:04.698901 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:33:04.698981 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:33:04.699269 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:33:04.699273 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:33:04.699275 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:33:04.699445 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:33:04.699552 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0809 09:33:04.699831 (+ 279us) server_negotiation.cc:299] Negotiation successful
0809 09:33:04.699943 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260809 09:33:04.701331 20982 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 09 09:33:04 dist-test-slave-7kzw krb5kdc[32272](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786267984, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 09:33:04 dist-test-slave-7kzw krb5kdc[32272](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.20.125.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 09:33:04 dist-test-slave-7kzw krb5kdc[32272](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.20.125.190@KRBTEST.COM, Server not found in Kerberos database
W20260809 09:33:04.727180 32409 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 09:33:04.720718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39910 (local address 127.20.125.190:43265)
0809 09:33:04.720861 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0809 09:33:04.720865 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 09:33:04.720878 (+ 13us) server_negotiation.cc:407] Connection header received
0809 09:33:04.720920 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:33:04.720923 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:33:04.720962 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:33:04.721035 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:33:04.721808 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:33:04.722411 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:33:04.723382 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:33:04.723645 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:33:04.723741 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:33:04.727065 (+ 3324us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39910: BlockingRecv error: recv got EOF from 127.0.0.1:39910 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 09 09:33:04 dist-test-slave-7kzw krb5kdc[32272](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786267984, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.20.125.190@KRBTEST.COM
I20260809 09:33:04.737005 32409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:33:04.728642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39918 (local address 127.20.125.190:43265)
0809 09:33:04.728803 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0809 09:33:04.728806 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 09:33:04.728870 (+ 64us) server_negotiation.cc:407] Connection header received
0809 09:33:04.728957 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:33:04.728960 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:33:04.728997 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:33:04.729061 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:33:04.729616 (+ 555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:33:04.730121 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:33:04.731054 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:33:04.731266 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:33:04.731351 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:33:04.733825 (+ 2474us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:33:04.733828 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:33:04.733830 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:33:04.733863 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:33:04.735443 (+ 1580us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:33:04.735863 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:33:04.735866 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:33:04.735868 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:33:04.735996 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:33:04.736260 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:33:04.736263 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:33:04.736264 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:33:04.736450 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:33:04.736561 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0809 09:33:04.736778 (+ 217us) server_negotiation.cc:299] Negotiation successful
0809 09:33:04.736876 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260809 09:33:04.740236 32336 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39918:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 09:33:04.743289 32336 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 09:33:04.759203 622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:33:04.752474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49832 (local address 127.20.125.129:40235)
0809 09:33:04.753115 (+ 641us) server_negotiation.cc:206] Beginning negotiation
0809 09:33:04.753126 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 09:33:04.753578 (+ 452us) server_negotiation.cc:407] Connection header received
0809 09:33:04.753871 (+ 293us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:33:04.753878 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:33:04.754090 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:33:04.754279 (+ 189us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 09:33:04.754584 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:33:04.755822 (+ 1238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:33:04.757906 (+ 2084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:33:04.758453 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:33:04.758647 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:33:04.758799 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0809 09:33:04.758933 (+ 134us) server_negotiation.cc:299] Negotiation successful
0809 09:33:04.759036 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":514,"thread_start_us":83,"threads_started":1}
I20260809 09:33:04.759361 623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:33:04.753069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.130:33845 (local address 127.0.0.1:56048)
0809 09:33:04.753816 (+ 747us) negotiation.cc:107] Waiting for socket to connect
0809 09:33:04.753830 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 09:33:04.753929 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:33:04.754445 (+ 516us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:33:04.754448 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:33:04.754506 (+ 58us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 09:33:04.754641 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:33:04.754648 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:33:04.756923 (+ 2275us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:33:04.756926 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:33:04.757818 (+ 892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:33:04.757829 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:33:04.758922 (+ 1093us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:33:04.758925 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:33:04.759005 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:33:04.759026 (+ 21us) client_negotiation.cc:769] Sending connection context
0809 09:33:04.759084 (+ 58us) client_negotiation.cc:240] Negotiation successful
0809 09:33:04.759194 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":663,"mutex_wait_us":81,"spinlock_wait_cycles":71808,"thread_start_us":133,"threads_started":1}
I20260809 09:33:04.759379 628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:33:04.753232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56048 (local address 127.20.125.130:33845)
0809 09:33:04.753618 (+ 386us) server_negotiation.cc:206] Beginning negotiation
0809 09:33:04.753630 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 09:33:04.753936 (+ 306us) server_negotiation.cc:407] Connection header received
0809 09:33:04.754027 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:33:04.754031 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:33:04.754189 (+ 158us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:33:04.754315 (+ 126us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 09:33:04.755836 (+ 1521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:33:04.756766 (+ 930us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:33:04.757955 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:33:04.758804 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:33:04.758947 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:33:04.759080 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0809 09:33:04.759131 (+ 51us) server_negotiation.cc:299] Negotiation successful
0809 09:33:04.759214 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":289,"thread_start_us":109,"threads_started":1}
I20260809 09:33:04.759979 626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:33:04.752355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.129:40235 (local address 127.0.0.1:49832)
0809 09:33:04.753430 (+ 1075us) negotiation.cc:107] Waiting for socket to connect
0809 09:33:04.753486 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0809 09:33:04.753673 (+ 187us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:33:04.754269 (+ 596us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:33:04.754275 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:33:04.754296 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 09:33:04.754429 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:33:04.754443 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:33:04.755965 (+ 1522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:33:04.755968 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:33:04.757797 (+ 1829us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:33:04.757805 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:33:04.758630 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:33:04.758633 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:33:04.758737 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:33:04.758774 (+ 37us) client_negotiation.cc:769] Sending connection context
0809 09:33:04.759764 (+ 990us) client_negotiation.cc:240] Negotiation successful
0809 09:33:04.759848 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":907,"mutex_wait_us":107,"spinlock_wait_cycles":5888,"thread_start_us":84,"threads_started":1}
I20260809 09:33:04.760169 620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:33:04.752565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.131:40493 (local address 127.0.0.1:38144)
0809 09:33:04.753430 (+ 865us) negotiation.cc:107] Waiting for socket to connect
0809 09:33:04.753486 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0809 09:33:04.753673 (+ 187us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:33:04.754445 (+ 772us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:33:04.754448 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:33:04.754503 (+ 55us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 09:33:04.754641 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:33:04.754648 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:33:04.755965 (+ 1317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:33:04.755968 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:33:04.757653 (+ 1685us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:33:04.757664 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:33:04.759772 (+ 2108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:33:04.759775 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:33:04.759841 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:33:04.759859 (+ 18us) client_negotiation.cc:769] Sending connection context
0809 09:33:04.759949 (+ 90us) client_negotiation.cc:240] Negotiation successful
0809 09:33:04.760027 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":766,"mutex_wait_us":194,"spinlock_wait_cycles":4480,"thread_start_us":95,"threads_started":1}
I20260809 09:33:04.760229 625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:33:04.752720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38144 (local address 127.20.125.131:40493)
0809 09:33:04.753618 (+ 898us) server_negotiation.cc:206] Beginning negotiation
0809 09:33:04.753630 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 09:33:04.753655 (+ 25us) server_negotiation.cc:407] Connection header received
0809 09:33:04.753936 (+ 281us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:33:04.753942 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:33:04.754135 (+ 193us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:33:04.754296 (+ 161us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 09:33:04.754764 (+ 468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:33:04.755841 (+ 1077us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:33:04.757809 (+ 1968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:33:04.758665 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:33:04.759562 (+ 897us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:33:04.759701 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0809 09:33:04.759998 (+ 297us) server_negotiation.cc:299] Negotiation successful
0809 09:33:04.760091 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":804,"thread_start_us":102,"threads_started":1}
I20260809 09:33:04.762840 32762 tablet_service.cc:1467] Processing CreateTablet for tablet 06b6d2113d5c4bdea6b878e94447543a (DEFAULT_TABLE table=test-table [id=c2a92353155747e8a9fa7f45f8e470a1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 09:33:04.762889 32545 tablet_service.cc:1467] Processing CreateTablet for tablet 06b6d2113d5c4bdea6b878e94447543a (DEFAULT_TABLE table=test-table [id=c2a92353155747e8a9fa7f45f8e470a1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 09:33:04.762900 487 tablet_service.cc:1467] Processing CreateTablet for tablet 06b6d2113d5c4bdea6b878e94447543a (DEFAULT_TABLE table=test-table [id=c2a92353155747e8a9fa7f45f8e470a1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 09:33:04.763978 487 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 06b6d2113d5c4bdea6b878e94447543a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 09:33:04.763973 32545 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 06b6d2113d5c4bdea6b878e94447543a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 09:33:04.764537 32762 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 06b6d2113d5c4bdea6b878e94447543a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 09:33:04.770539 630 tablet_bootstrap.cc:492] T 06b6d2113d5c4bdea6b878e94447543a P b44b10e0ca7c4c9e89325813a36288d7: Bootstrap starting.
I20260809 09:33:04.771948 631 tablet_bootstrap.cc:492] T 06b6d2113d5c4bdea6b878e94447543a P 2313d46fd7f84023b99b769f25a2bd37: Bootstrap starting.
I20260809 09:33:04.772895 630 tablet_bootstrap.cc:654] T 06b6d2113d5c4bdea6b878e94447543a P b44b10e0ca7c4c9e89325813a36288d7: Neither blocks nor log segments found. Creating new log.
I20260809 09:33:04.773618 630 log.cc:824] T 06b6d2113d5c4bdea6b878e94447543a P b44b10e0ca7c4c9e89325813a36288d7: Log is configured to *not* fsync() on all Append() calls
I20260809 09:33:04.773759 631 tablet_bootstrap.cc:654] T 06b6d2113d5c4bdea6b878e94447543a P 2313d46fd7f84023b99b769f25a2bd37: Neither blocks nor log segments found. Creating new log.
I20260809 09:33:04.774456 631 log.cc:824] T 06b6d2113d5c4bdea6b878e94447543a P 2313d46fd7f84023b99b769f25a2bd37: Log is configured to *not* fsync() on all Append() calls
I20260809 09:33:04.774798 632 tablet_bootstrap.cc:492] T 06b6d2113d5c4bdea6b878e94447543a P 2a843b4644cd4b2389cf749e5c3f2e58: Bootstrap starting.
I20260809 09:33:04.775789 630 tablet_bootstrap.cc:492] T 06b6d2113d5c4bdea6b878e94447543a P b44b10e0ca7c4c9e89325813a36288d7: No bootstrap required, opened a new log
I20260809 09:33:04.775983 630 ts_tablet_manager.cc:1397] T 06b6d2113d5c4bdea6b878e94447543a P b44b10e0ca7c4c9e89325813a36288d7: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260809 09:33:04.776760 631 tablet_bootstrap.cc:492] T 06b6d2113d5c4bdea6b878e94447543a P 2313d46fd7f84023b99b769f25a2bd37: No bootstrap required, opened a new log
I20260809 09:33:04.776815 632 tablet_bootstrap.cc:654] T 06b6d2113d5c4bdea6b878e94447543a P 2a843b4644cd4b2389cf749e5c3f2e58: Neither blocks nor log segments found. Creating new log.
I20260809 09:33:04.776949 631 ts_tablet_manager.cc:1397] T 06b6d2113d5c4bdea6b878e94447543a P 2313d46fd7f84023b99b769f25a2bd37: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260809 09:33:04.777599 632 log.cc:824] T 06b6d2113d5c4bdea6b878e94447543a P 2a843b4644cd4b2389cf749e5c3f2e58: Log is configured to *not* fsync() on all Append() calls
I20260809 09:33:04.779736 632 tablet_bootstrap.cc:492] T 06b6d2113d5c4bdea6b878e94447543a P 2a843b4644cd4b2389cf749e5c3f2e58: No bootstrap required, opened a new log
I20260809 09:33:04.779623 630 raft_consensus.cc:348] T 06b6d2113d5c4bdea6b878e94447543a P b44b10e0ca7c4c9e89325813a36288d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a843b4644cd4b2389cf749e5c3f2e58" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 40235 } } peers { permanent_uuid: "b44b10e0ca7c4c9e89325813a36288d7" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 33845 } } peers { permanent_uuid: "2313d46fd7f84023b99b769f25a2bd37" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 40493 } }
I20260809 09:33:04.779904 630 raft_consensus.cc:374] T 06b6d2113d5c4bdea6b878e94447543a P b44b10e0ca7c4c9e89325813a36288d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 09:33:04.779913 632 ts_tablet_manager.cc:1397] T 06b6d2113d5c4bdea6b878e94447543a P 2a843b4644cd4b2389cf749e5c3f2e58: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260809 09:33:04.779963 630 raft_consensus.cc:729] T 06b6d2113d5c4bdea6b878e94447543a P b44b10e0ca7c4c9e89325813a36288d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b44b10e0ca7c4c9e89325813a36288d7, State: Initialized, Role: FOLLOWER
I20260809 09:33:04.780435 630 consensus_queue.cc:260] T 06b6d2113d5c4bdea6b878e94447543a P b44b10e0ca7c4c9e89325813a36288d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a843b4644cd4b2389cf749e5c3f2e58" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 40235 } } peers { permanent_uuid: "b44b10e0ca7c4c9e89325813a36288d7" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 33845 } } peers { permanent_uuid: "2313d46fd7f84023b99b769f25a2bd37" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 40493 } }
I20260809 09:33:04.780735 631 raft_consensus.cc:348] T 06b6d2113d5c4bdea6b878e94447543a P 2313d46fd7f84023b99b769f25a2bd37 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a843b4644cd4b2389cf749e5c3f2e58" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 40235 } } peers { permanent_uuid: "b44b10e0ca7c4c9e89325813a36288d7" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 33845 } } peers { permanent_uuid: "2313d46fd7f84023b99b769f25a2bd37" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 40493 } }
I20260809 09:33:04.781140 387 heartbeater.cc:503] Master 127.20.125.190:43265 was elected leader, sending a full tablet report...
I20260809 09:33:04.781451 630 ts_tablet_manager.cc:1428] T 06b6d2113d5c4bdea6b878e94447543a P b44b10e0ca7c4c9e89325813a36288d7: Time spent starting tablet: real 0.005s user 0.003s sys 0.003s
I20260809 09:33:04.781001 631 raft_consensus.cc:374] T 06b6d2113d5c4bdea6b878e94447543a P 2313d46fd7f84023b99b769f25a2bd37 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 09:33:04.781775 631 raft_consensus.cc:729] T 06b6d2113d5c4bdea6b878e94447543a P 2313d46fd7f84023b99b769f25a2bd37 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2313d46fd7f84023b99b769f25a2bd37, State: Initialized, Role: FOLLOWER
I20260809 09:33:04.781962 642 raft_consensus.cc:482] T 06b6d2113d5c4bdea6b878e94447543a P b44b10e0ca7c4c9e89325813a36288d7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 09:33:04.782217 631 consensus_queue.cc:260] T 06b6d2113d5c4bdea6b878e94447543a P 2313d46fd7f84023b99b769f25a2bd37 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a843b4644cd4b2389cf749e5c3f2e58" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 40235 } } peers { permanent_uuid: "b44b10e0ca7c4c9e89325813a36288d7" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 33845 } } peers { permanent_uuid: "2313d46fd7f84023b99b769f25a2bd37" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 40493 } }
I20260809 09:33:04.782294 642 raft_consensus.cc:504] T 06b6d2113d5c4bdea6b878e94447543a P b44b10e0ca7c4c9e89325813a36288d7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a843b4644cd4b2389cf749e5c3f2e58" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 40235 } } peers { permanent_uuid: "b44b10e0ca7c4c9e89325813a36288d7" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 33845 } } peers { permanent_uuid: "2313d46fd7f84023b99b769f25a2bd37" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 40493 } }
I20260809 09:33:04.783221 631 ts_tablet_manager.cc:1428] T 06b6d2113d5c4bdea6b878e94447543a P 2313d46fd7f84023b99b769f25a2bd37: Time spent starting tablet: real 0.006s user 0.005s sys 0.001s
I20260809 09:33:04.783548 585 heartbeater.cc:503] Master 127.20.125.190:43265 was elected leader, sending a full tablet report...
I20260809 09:33:04.784476 632 raft_consensus.cc:348] T 06b6d2113d5c4bdea6b878e94447543a P 2a843b4644cd4b2389cf749e5c3f2e58 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a843b4644cd4b2389cf749e5c3f2e58" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 40235 } } peers { permanent_uuid: "b44b10e0ca7c4c9e89325813a36288d7" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 33845 } } peers { permanent_uuid: "2313d46fd7f84023b99b769f25a2bd37" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 40493 } }
I20260809 09:33:04.784739 632 raft_consensus.cc:374] T 06b6d2113d5c4bdea6b878e94447543a P 2a843b4644cd4b2389cf749e5c3f2e58 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 09:33:04.784795 632 raft_consensus.cc:729] T 06b6d2113d5c4bdea6b878e94447543a P 2a843b4644cd4b2389cf749e5c3f2e58 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a843b4644cd4b2389cf749e5c3f2e58, State: Initialized, Role: FOLLOWER
I20260809 09:33:04.785830 642 leader_election.cc:288] T 06b6d2113d5c4bdea6b878e94447543a P b44b10e0ca7c4c9e89325813a36288d7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2a843b4644cd4b2389cf749e5c3f2e58 (127.20.125.129:40235), 2313d46fd7f84023b99b769f25a2bd37 (127.20.125.131:40493)
I20260809 09:33:04.785262 632 consensus_queue.cc:260] T 06b6d2113d5c4bdea6b878e94447543a P 2a843b4644cd4b2389cf749e5c3f2e58 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a843b4644cd4b2389cf749e5c3f2e58" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 40235 } } peers { permanent_uuid: "b44b10e0ca7c4c9e89325813a36288d7" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 33845 } } peers { permanent_uuid: "2313d46fd7f84023b99b769f25a2bd37" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 40493 } }
I20260809 09:33:04.787102 32647 heartbeater.cc:503] Master 127.20.125.190:43265 was elected leader, sending a full tablet report...
I20260809 09:33:04.787416 632 ts_tablet_manager.cc:1428] T 06b6d2113d5c4bdea6b878e94447543a P 2a843b4644cd4b2389cf749e5c3f2e58: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260809 09:33:04.789331 622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:33:04.784385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.130:58911 (local address 127.20.125.129:40235)
0809 09:33:04.784509 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0809 09:33:04.784514 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 09:33:04.784528 (+ 14us) server_negotiation.cc:407] Connection header received
0809 09:33:04.784574 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:33:04.784577 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:33:04.784618 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:33:04.784696 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 09:33:04.785166 (+ 470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:33:04.785933 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:33:04.787878 (+ 1945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:33:04.788488 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:33:04.788625 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:33:04.788702 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0809 09:33:04.789146 (+ 444us) server_negotiation.cc:299] Negotiation successful
0809 09:33:04.789217 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 09:33:04.789345 32698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:33:04.783787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.129:40235 (local address 127.20.125.130:58911)
0809 09:33:04.784082 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0809 09:33:04.784097 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 09:33:04.784177 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:33:04.784902 (+ 725us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:33:04.784905 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:33:04.784924 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 09:33:04.785034 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:33:04.785041 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:33:04.786103 (+ 1062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:33:04.786106 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:33:04.787715 (+ 1609us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:33:04.787726 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:33:04.788879 (+ 1153us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:33:04.788884 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:33:04.788995 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:33:04.789020 (+ 25us) client_negotiation.cc:769] Sending connection context
0809 09:33:04.789130 (+ 110us) client_negotiation.cc:240] Negotiation successful
0809 09:33:04.789204 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":219,"mutex_wait_us":447}
I20260809 09:33:04.789934 32577 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06b6d2113d5c4bdea6b878e94447543a" candidate_uuid: "b44b10e0ca7c4c9e89325813a36288d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a843b4644cd4b2389cf749e5c3f2e58" is_pre_election: true
I20260809 09:33:04.790241 32577 raft_consensus.cc:2454] T 06b6d2113d5c4bdea6b878e94447543a P 2a843b4644cd4b2389cf749e5c3f2e58 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b44b10e0ca7c4c9e89325813a36288d7 in term 0.
I20260809 09:33:04.790773 32685 leader_election.cc:302] T 06b6d2113d5c4bdea6b878e94447543a P b44b10e0ca7c4c9e89325813a36288d7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a843b4644cd4b2389cf749e5c3f2e58, b44b10e0ca7c4c9e89325813a36288d7; no voters:
I20260809 09:33:04.791074 644 raft_consensus.cc:2781] T 06b6d2113d5c4bdea6b878e94447543a P b44b10e0ca7c4c9e89325813a36288d7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 09:33:04.791167 644 raft_consensus.cc:482] T 06b6d2113d5c4bdea6b878e94447543a P b44b10e0ca7c4c9e89325813a36288d7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 09:33:04.791230 644 raft_consensus.cc:3037] T 06b6d2113d5c4bdea6b878e94447543a P b44b10e0ca7c4c9e89325813a36288d7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 09:33:04.791662 625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:33:04.785888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.130:40953 (local address 127.20.125.131:40493)
0809 09:33:04.786011 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0809 09:33:04.786029 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0809 09:33:04.786313 (+ 284us) server_negotiation.cc:407] Connection header received
0809 09:33:04.786423 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:33:04.786426 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:33:04.786496 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:33:04.786578 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 09:33:04.788237 (+ 1659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:33:04.788933 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:33:04.790531 (+ 1598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:33:04.791127 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:33:04.791259 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:33:04.791305 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0809 09:33:04.791486 (+ 181us) server_negotiation.cc:299] Negotiation successful
0809 09:33:04.791553 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260809 09:33:04.791692 648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:33:04.785543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.131:40493 (local address 127.20.125.130:40953)
0809 09:33:04.786213 (+ 670us) negotiation.cc:107] Waiting for socket to connect
0809 09:33:04.786229 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 09:33:04.786325 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:33:04.786676 (+ 351us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:33:04.786679 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:33:04.786695 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 09:33:04.788086 (+ 1391us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:33:04.788096 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:33:04.789083 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:33:04.789086 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:33:04.790371 (+ 1285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:33:04.790384 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:33:04.791254 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:33:04.791257 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:33:04.791355 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:33:04.791376 (+ 21us) client_negotiation.cc:769] Sending connection context
0809 09:33:04.791472 (+ 96us) client_negotiation.cc:240] Negotiation successful
0809 09:33:04.791540 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":594,"mutex_wait_us":3,"thread_start_us":88,"threads_started":1}
I20260809 09:33:04.792193 519 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06b6d2113d5c4bdea6b878e94447543a" candidate_uuid: "b44b10e0ca7c4c9e89325813a36288d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2313d46fd7f84023b99b769f25a2bd37" is_pre_election: true
I20260809 09:33:04.792387 519 raft_consensus.cc:2454] T 06b6d2113d5c4bdea6b878e94447543a P 2313d46fd7f84023b99b769f25a2bd37 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b44b10e0ca7c4c9e89325813a36288d7 in term 0.
I20260809 09:33:04.792518 644 raft_consensus.cc:504] T 06b6d2113d5c4bdea6b878e94447543a P b44b10e0ca7c4c9e89325813a36288d7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a843b4644cd4b2389cf749e5c3f2e58" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 40235 } } peers { permanent_uuid: "b44b10e0ca7c4c9e89325813a36288d7" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 33845 } } peers { permanent_uuid: "2313d46fd7f84023b99b769f25a2bd37" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 40493 } }
I20260809 09:33:04.792858 644 leader_election.cc:288] T 06b6d2113d5c4bdea6b878e94447543a P b44b10e0ca7c4c9e89325813a36288d7 [CANDIDATE]: Term 1 election: Requested vote from peers 2a843b4644cd4b2389cf749e5c3f2e58 (127.20.125.129:40235), 2313d46fd7f84023b99b769f25a2bd37 (127.20.125.131:40493)
I20260809 09:33:04.793190 519 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06b6d2113d5c4bdea6b878e94447543a" candidate_uuid: "b44b10e0ca7c4c9e89325813a36288d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2313d46fd7f84023b99b769f25a2bd37"
I20260809 09:33:04.793200 32577 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06b6d2113d5c4bdea6b878e94447543a" candidate_uuid: "b44b10e0ca7c4c9e89325813a36288d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a843b4644cd4b2389cf749e5c3f2e58"
I20260809 09:33:04.793301 32577 raft_consensus.cc:3037] T 06b6d2113d5c4bdea6b878e94447543a P 2a843b4644cd4b2389cf749e5c3f2e58 [term 0 FOLLOWER]: Advancing to term 1
I20260809 09:33:04.793313 519 raft_consensus.cc:3037] T 06b6d2113d5c4bdea6b878e94447543a P 2313d46fd7f84023b99b769f25a2bd37 [term 0 FOLLOWER]: Advancing to term 1
I20260809 09:33:04.794279 32577 raft_consensus.cc:2454] T 06b6d2113d5c4bdea6b878e94447543a P 2a843b4644cd4b2389cf749e5c3f2e58 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b44b10e0ca7c4c9e89325813a36288d7 in term 1.
I20260809 09:33:04.794296 519 raft_consensus.cc:2454] T 06b6d2113d5c4bdea6b878e94447543a P 2313d46fd7f84023b99b769f25a2bd37 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b44b10e0ca7c4c9e89325813a36288d7 in term 1.
I20260809 09:33:04.794591 32685 leader_election.cc:302] T 06b6d2113d5c4bdea6b878e94447543a P b44b10e0ca7c4c9e89325813a36288d7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a843b4644cd4b2389cf749e5c3f2e58, b44b10e0ca7c4c9e89325813a36288d7; no voters:
I20260809 09:33:04.794751 644 raft_consensus.cc:2781] T 06b6d2113d5c4bdea6b878e94447543a P b44b10e0ca7c4c9e89325813a36288d7 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 09:33:04.794849 644 raft_consensus.cc:686] T 06b6d2113d5c4bdea6b878e94447543a P b44b10e0ca7c4c9e89325813a36288d7 [term 1 LEADER]: Becoming Leader. State: Replica: b44b10e0ca7c4c9e89325813a36288d7, State: Running, Role: LEADER
I20260809 09:33:04.795193 644 consensus_queue.cc:237] T 06b6d2113d5c4bdea6b878e94447543a P b44b10e0ca7c4c9e89325813a36288d7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a843b4644cd4b2389cf749e5c3f2e58" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 40235 } } peers { permanent_uuid: "b44b10e0ca7c4c9e89325813a36288d7" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 33845 } } peers { permanent_uuid: "2313d46fd7f84023b99b769f25a2bd37" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 40493 } }
I20260809 09:33:04.798413 32340 catalog_manager.cc:5495] T 06b6d2113d5c4bdea6b878e94447543a P b44b10e0ca7c4c9e89325813a36288d7 reported cstate change: term changed from 0 to 1, leader changed from <none> to b44b10e0ca7c4c9e89325813a36288d7 (127.20.125.130). New cstate: current_term: 1 leader_uuid: "b44b10e0ca7c4c9e89325813a36288d7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a843b4644cd4b2389cf749e5c3f2e58" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 40235 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b44b10e0ca7c4c9e89325813a36288d7" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 33845 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2313d46fd7f84023b99b769f25a2bd37" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 40493 } health_report { overall_health: UNKNOWN } } }
I20260809 09:33:04.817075 628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:33:04.813406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56058 (local address 127.20.125.130:33845)
0809 09:33:04.813549 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0809 09:33:04.813553 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 09:33:04.813566 (+ 13us) server_negotiation.cc:407] Connection header received
0809 09:33:04.813640 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:33:04.813642 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:33:04.813693 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:33:04.813789 (+ 96us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 09:33:04.814068 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:33:04.814683 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:33:04.815575 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:33:04.815815 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:33:04.815903 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:33:04.816229 (+ 326us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 09:33:04.816689 (+ 460us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 09:33:04.816784 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0809 09:33:04.816892 (+ 108us) server_negotiation.cc:299] Negotiation successful
0809 09:33:04.816947 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260809 09:33:04.823220 32577 raft_consensus.cc:1261] T 06b6d2113d5c4bdea6b878e94447543a P 2a843b4644cd4b2389cf749e5c3f2e58 [term 1 FOLLOWER]: Refusing update from remote peer b44b10e0ca7c4c9e89325813a36288d7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 09:33:04.823388 519 raft_consensus.cc:1261] T 06b6d2113d5c4bdea6b878e94447543a P 2313d46fd7f84023b99b769f25a2bd37 [term 1 FOLLOWER]: Refusing update from remote peer b44b10e0ca7c4c9e89325813a36288d7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 09:33:04.824102 642 consensus_queue.cc:1035] T 06b6d2113d5c4bdea6b878e94447543a P b44b10e0ca7c4c9e89325813a36288d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2313d46fd7f84023b99b769f25a2bd37" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 40493 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 09:33:04.824317 644 consensus_queue.cc:1035] T 06b6d2113d5c4bdea6b878e94447543a P b44b10e0ca7c4c9e89325813a36288d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2a843b4644cd4b2389cf749e5c3f2e58" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 40235 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 09:33:04.832933 660 mvcc.cc:205] Tried to move back new op lower bound from 7316553665827725312 to 7316553665723846656. Current Snapshot: MvccSnapshot[applied={T|T < 7316553665827725312}]
I20260809 09:33:04.833156 662 mvcc.cc:205] Tried to move back new op lower bound from 7316553665827725312 to 7316553665723846656. Current Snapshot: MvccSnapshot[applied={T|T < 7316553665827725312}]
I20260809 09:33:04.841538 32340 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39918:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 09:33:04.841724 32340 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39918:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 09:33:04.844151 32340 catalog_manager.cc:5782] T 00000000000000000000000000000000 P fdd08fcf074d4c108a4ea936f91b202d: Sending DeleteTablet for 3 replicas of tablet 06b6d2113d5c4bdea6b878e94447543a
I20260809 09:33:04.845026 32545 tablet_service.cc:1514] Processing DeleteTablet for tablet 06b6d2113d5c4bdea6b878e94447543a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 09:33:04 UTC) from {username='oryx', principal='oryx/127.20.125.190@KRBTEST.COM'} at 127.0.0.1:49832
I20260809 09:33:04.845098 487 tablet_service.cc:1514] Processing DeleteTablet for tablet 06b6d2113d5c4bdea6b878e94447543a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 09:33:04 UTC) from {username='oryx', principal='oryx/127.20.125.190@KRBTEST.COM'} at 127.0.0.1:38144
I20260809 09:33:04.845758 32340 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39918:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 09:33:04.845856 674 tablet_replica.cc:330] T 06b6d2113d5c4bdea6b878e94447543a P 2a843b4644cd4b2389cf749e5c3f2e58: stopping tablet replica
I20260809 09:33:04.845810 32762 tablet_service.cc:1514] Processing DeleteTablet for tablet 06b6d2113d5c4bdea6b878e94447543a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 09:33:04 UTC) from {username='oryx', principal='oryx/127.20.125.190@KRBTEST.COM'} at 127.0.0.1:56048
I20260809 09:33:04.846167 674 raft_consensus.cc:2229] T 06b6d2113d5c4bdea6b878e94447543a P 2a843b4644cd4b2389cf749e5c3f2e58 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260809 09:33:04.846216 32340 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 09:33:04.846511 674 raft_consensus.cc:2258] T 06b6d2113d5c4bdea6b878e94447543a P 2a843b4644cd4b2389cf749e5c3f2e58 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 09:33:04.846562 675 tablet_replica.cc:330] T 06b6d2113d5c4bdea6b878e94447543a P b44b10e0ca7c4c9e89325813a36288d7: stopping tablet replica
I20260809 09:33:04.847082 675 raft_consensus.cc:2229] T 06b6d2113d5c4bdea6b878e94447543a P b44b10e0ca7c4c9e89325813a36288d7 [term 1 LEADER]: Raft consensus shutting down.
I20260809 09:33:04.847122 673 tablet_replica.cc:330] T 06b6d2113d5c4bdea6b878e94447543a P 2313d46fd7f84023b99b769f25a2bd37: stopping tablet replica
I20260809 09:33:04.847412 673 raft_consensus.cc:2229] T 06b6d2113d5c4bdea6b878e94447543a P 2313d46fd7f84023b99b769f25a2bd37 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 09:33:04.847682 675 raft_consensus.cc:2258] T 06b6d2113d5c4bdea6b878e94447543a P b44b10e0ca7c4c9e89325813a36288d7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 09:33:04.847713 673 raft_consensus.cc:2258] T 06b6d2113d5c4bdea6b878e94447543a P 2313d46fd7f84023b99b769f25a2bd37 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 09:33:04.848285 674 ts_tablet_manager.cc:1905] T 06b6d2113d5c4bdea6b878e94447543a P 2a843b4644cd4b2389cf749e5c3f2e58: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 09:33:04.849701 675 ts_tablet_manager.cc:1905] T 06b6d2113d5c4bdea6b878e94447543a P b44b10e0ca7c4c9e89325813a36288d7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 09:33:04.851310 674 ts_tablet_manager.cc:1918] T 06b6d2113d5c4bdea6b878e94447543a P 2a843b4644cd4b2389cf749e5c3f2e58: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 09:33:04.851420 674 log.cc:1196] T 06b6d2113d5c4bdea6b878e94447543a P 2a843b4644cd4b2389cf749e5c3f2e58: Deleting WAL directory at /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786267941944493-20982-0/minicluster-data/ts-0/wal/wals/06b6d2113d5c4bdea6b878e94447543a
I20260809 09:33:04.851624 32762 tablet_service.cc:1467] Processing CreateTablet for tablet 9ea69a45821c40099c23e6d5323a500b (DEFAULT_TABLE table=test-table [id=f3a12d6c177347ea83fcaf9d2804aa05]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 09:33:04.851632 32545 tablet_service.cc:1467] Processing CreateTablet for tablet 9ea69a45821c40099c23e6d5323a500b (DEFAULT_TABLE table=test-table [id=f3a12d6c177347ea83fcaf9d2804aa05]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 09:33:04.851763 674 ts_tablet_manager.cc:1939] T 06b6d2113d5c4bdea6b878e94447543a P 2a843b4644cd4b2389cf749e5c3f2e58: Deleting consensus metadata
I20260809 09:33:04.851790 673 ts_tablet_manager.cc:1905] T 06b6d2113d5c4bdea6b878e94447543a P 2313d46fd7f84023b99b769f25a2bd37: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 09:33:04.851951 32545 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9ea69a45821c40099c23e6d5323a500b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 09:33:04.851962 32762 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9ea69a45821c40099c23e6d5323a500b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 09:33:04.853009 32316 catalog_manager.cc:4841] TS 2a843b4644cd4b2389cf749e5c3f2e58 (127.20.125.129:40235): tablet 06b6d2113d5c4bdea6b878e94447543a (table test-table [id=c2a92353155747e8a9fa7f45f8e470a1]) successfully deleted
I20260809 09:33:04.854142 630 tablet_bootstrap.cc:492] T 9ea69a45821c40099c23e6d5323a500b P b44b10e0ca7c4c9e89325813a36288d7: Bootstrap starting.
I20260809 09:33:04.854259 632 tablet_bootstrap.cc:492] T 9ea69a45821c40099c23e6d5323a500b P 2a843b4644cd4b2389cf749e5c3f2e58: Bootstrap starting.
I20260809 09:33:04.854449 673 ts_tablet_manager.cc:1918] T 06b6d2113d5c4bdea6b878e94447543a P 2313d46fd7f84023b99b769f25a2bd37: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 09:33:04.854588 673 log.cc:1196] T 06b6d2113d5c4bdea6b878e94447543a P 2313d46fd7f84023b99b769f25a2bd37: Deleting WAL directory at /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786267941944493-20982-0/minicluster-data/ts-2/wal/wals/06b6d2113d5c4bdea6b878e94447543a
I20260809 09:33:04.854923 673 ts_tablet_manager.cc:1939] T 06b6d2113d5c4bdea6b878e94447543a P 2313d46fd7f84023b99b769f25a2bd37: Deleting consensus metadata
I20260809 09:33:04.855175 630 tablet_bootstrap.cc:654] T 9ea69a45821c40099c23e6d5323a500b P b44b10e0ca7c4c9e89325813a36288d7: Neither blocks nor log segments found. Creating new log.
I20260809 09:33:04.855335 487 tablet_service.cc:1467] Processing CreateTablet for tablet 9ea69a45821c40099c23e6d5323a500b (DEFAULT_TABLE table=test-table [id=f3a12d6c177347ea83fcaf9d2804aa05]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 09:33:04.855504 632 tablet_bootstrap.cc:654] T 9ea69a45821c40099c23e6d5323a500b P 2a843b4644cd4b2389cf749e5c3f2e58: Neither blocks nor log segments found. Creating new log.
I20260809 09:33:04.855568 487 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9ea69a45821c40099c23e6d5323a500b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 09:33:04.856098 32318 catalog_manager.cc:4841] TS 2313d46fd7f84023b99b769f25a2bd37 (127.20.125.131:40493): tablet 06b6d2113d5c4bdea6b878e94447543a (table test-table [id=c2a92353155747e8a9fa7f45f8e470a1]) successfully deleted
I20260809 09:33:04.857141 675 ts_tablet_manager.cc:1918] T 06b6d2113d5c4bdea6b878e94447543a P b44b10e0ca7c4c9e89325813a36288d7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 09:33:04.857234 675 log.cc:1196] T 06b6d2113d5c4bdea6b878e94447543a P b44b10e0ca7c4c9e89325813a36288d7: Deleting WAL directory at /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786267941944493-20982-0/minicluster-data/ts-1/wal/wals/06b6d2113d5c4bdea6b878e94447543a
I20260809 09:33:04.857543 675 ts_tablet_manager.cc:1939] T 06b6d2113d5c4bdea6b878e94447543a P b44b10e0ca7c4c9e89325813a36288d7: Deleting consensus metadata
I20260809 09:33:04.857651 631 tablet_bootstrap.cc:492] T 9ea69a45821c40099c23e6d5323a500b P 2313d46fd7f84023b99b769f25a2bd37: Bootstrap starting.
I20260809 09:33:04.858522 631 tablet_bootstrap.cc:654] T 9ea69a45821c40099c23e6d5323a500b P 2313d46fd7f84023b99b769f25a2bd37: Neither blocks nor log segments found. Creating new log.
I20260809 09:33:04.858606 32316 catalog_manager.cc:4841] TS b44b10e0ca7c4c9e89325813a36288d7 (127.20.125.130:33845): tablet 06b6d2113d5c4bdea6b878e94447543a (table test-table [id=c2a92353155747e8a9fa7f45f8e470a1]) successfully deleted
I20260809 09:33:04.859347 632 tablet_bootstrap.cc:492] T 9ea69a45821c40099c23e6d5323a500b P 2a843b4644cd4b2389cf749e5c3f2e58: No bootstrap required, opened a new log
I20260809 09:33:04.859441 632 ts_tablet_manager.cc:1397] T 9ea69a45821c40099c23e6d5323a500b P 2a843b4644cd4b2389cf749e5c3f2e58: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260809 09:33:04.859565 630 tablet_bootstrap.cc:492] T 9ea69a45821c40099c23e6d5323a500b P b44b10e0ca7c4c9e89325813a36288d7: No bootstrap required, opened a new log
I20260809 09:33:04.859676 630 ts_tablet_manager.cc:1397] T 9ea69a45821c40099c23e6d5323a500b P b44b10e0ca7c4c9e89325813a36288d7: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20260809 09:33:04.859947 632 raft_consensus.cc:348] T 9ea69a45821c40099c23e6d5323a500b P 2a843b4644cd4b2389cf749e5c3f2e58 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a843b4644cd4b2389cf749e5c3f2e58" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 40235 } } peers { permanent_uuid: "b44b10e0ca7c4c9e89325813a36288d7" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 33845 } } peers { permanent_uuid: "2313d46fd7f84023b99b769f25a2bd37" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 40493 } }
I20260809 09:33:04.860073 632 raft_consensus.cc:374] T 9ea69a45821c40099c23e6d5323a500b P 2a843b4644cd4b2389cf749e5c3f2e58 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 09:33:04.860119 632 raft_consensus.cc:729] T 9ea69a45821c40099c23e6d5323a500b P 2a843b4644cd4b2389cf749e5c3f2e58 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a843b4644cd4b2389cf749e5c3f2e58, State: Initialized, Role: FOLLOWER
I20260809 09:33:04.860096 630 raft_consensus.cc:348] T 9ea69a45821c40099c23e6d5323a500b P b44b10e0ca7c4c9e89325813a36288d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a843b4644cd4b2389cf749e5c3f2e58" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 40235 } } peers { permanent_uuid: "b44b10e0ca7c4c9e89325813a36288d7" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 33845 } } peers { permanent_uuid: "2313d46fd7f84023b99b769f25a2bd37" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 40493 } }
I20260809 09:33:04.860208 630 raft_consensus.cc:374] T 9ea69a45821c40099c23e6d5323a500b P b44b10e0ca7c4c9e89325813a36288d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 09:33:04.860250 630 raft_consensus.cc:729] T 9ea69a45821c40099c23e6d5323a500b P b44b10e0ca7c4c9e89325813a36288d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b44b10e0ca7c4c9e89325813a36288d7, State: Initialized, Role: FOLLOWER
I20260809 09:33:04.860265 632 consensus_queue.cc:260] T 9ea69a45821c40099c23e6d5323a500b P 2a843b4644cd4b2389cf749e5c3f2e58 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a843b4644cd4b2389cf749e5c3f2e58" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 40235 } } peers { permanent_uuid: "b44b10e0ca7c4c9e89325813a36288d7" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 33845 } } peers { permanent_uuid: "2313d46fd7f84023b99b769f25a2bd37" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 40493 } }
I20260809 09:33:04.860374 630 consensus_queue.cc:260] T 9ea69a45821c40099c23e6d5323a500b P b44b10e0ca7c4c9e89325813a36288d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a843b4644cd4b2389cf749e5c3f2e58" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 40235 } } peers { permanent_uuid: "b44b10e0ca7c4c9e89325813a36288d7" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 33845 } } peers { permanent_uuid: "2313d46fd7f84023b99b769f25a2bd37" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 40493 } }
I20260809 09:33:04.860553 632 ts_tablet_manager.cc:1428] T 9ea69a45821c40099c23e6d5323a500b P 2a843b4644cd4b2389cf749e5c3f2e58: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260809 09:33:04.860638 630 ts_tablet_manager.cc:1428] T 9ea69a45821c40099c23e6d5323a500b P b44b10e0ca7c4c9e89325813a36288d7: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260809 09:33:04.861515 631 tablet_bootstrap.cc:492] T 9ea69a45821c40099c23e6d5323a500b P 2313d46fd7f84023b99b769f25a2bd37: No bootstrap required, opened a new log
I20260809 09:33:04.861649 631 ts_tablet_manager.cc:1397] T 9ea69a45821c40099c23e6d5323a500b P 2313d46fd7f84023b99b769f25a2bd37: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.001s
I20260809 09:33:04.862053 631 raft_consensus.cc:348] T 9ea69a45821c40099c23e6d5323a500b P 2313d46fd7f84023b99b769f25a2bd37 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a843b4644cd4b2389cf749e5c3f2e58" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 40235 } } peers { permanent_uuid: "b44b10e0ca7c4c9e89325813a36288d7" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 33845 } } peers { permanent_uuid: "2313d46fd7f84023b99b769f25a2bd37" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 40493 } }
I20260809 09:33:04.862169 631 raft_consensus.cc:374] T 9ea69a45821c40099c23e6d5323a500b P 2313d46fd7f84023b99b769f25a2bd37 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 09:33:04.862238 631 raft_consensus.cc:729] T 9ea69a45821c40099c23e6d5323a500b P 2313d46fd7f84023b99b769f25a2bd37 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2313d46fd7f84023b99b769f25a2bd37, State: Initialized, Role: FOLLOWER
I20260809 09:33:04.862381 631 consensus_queue.cc:260] T 9ea69a45821c40099c23e6d5323a500b P 2313d46fd7f84023b99b769f25a2bd37 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a843b4644cd4b2389cf749e5c3f2e58" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 40235 } } peers { permanent_uuid: "b44b10e0ca7c4c9e89325813a36288d7" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 33845 } } peers { permanent_uuid: "2313d46fd7f84023b99b769f25a2bd37" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 40493 } }
I20260809 09:33:04.862675 631 ts_tablet_manager.cc:1428] T 9ea69a45821c40099c23e6d5323a500b P 2313d46fd7f84023b99b769f25a2bd37: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
W20260809 09:33:04.938349 587 tablet.cc:2369] T 9ea69a45821c40099c23e6d5323a500b P 2313d46fd7f84023b99b769f25a2bd37: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 09:33:04.954530 389 tablet.cc:2369] T 9ea69a45821c40099c23e6d5323a500b P b44b10e0ca7c4c9e89325813a36288d7: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 09:33:04.983075 32648 tablet.cc:2369] T 9ea69a45821c40099c23e6d5323a500b P 2a843b4644cd4b2389cf749e5c3f2e58: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 09:33:05.030248 667 raft_consensus.cc:482] T 9ea69a45821c40099c23e6d5323a500b P b44b10e0ca7c4c9e89325813a36288d7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 09:33:05.030441 667 raft_consensus.cc:504] T 9ea69a45821c40099c23e6d5323a500b P b44b10e0ca7c4c9e89325813a36288d7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a843b4644cd4b2389cf749e5c3f2e58" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 40235 } } peers { permanent_uuid: "b44b10e0ca7c4c9e89325813a36288d7" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 33845 } } peers { permanent_uuid: "2313d46fd7f84023b99b769f25a2bd37" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 40493 } }
I20260809 09:33:05.031064 667 leader_election.cc:288] T 9ea69a45821c40099c23e6d5323a500b P b44b10e0ca7c4c9e89325813a36288d7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2a843b4644cd4b2389cf749e5c3f2e58 (127.20.125.129:40235), 2313d46fd7f84023b99b769f25a2bd37 (127.20.125.131:40493)
I20260809 09:33:05.031615 32577 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ea69a45821c40099c23e6d5323a500b" candidate_uuid: "b44b10e0ca7c4c9e89325813a36288d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a843b4644cd4b2389cf749e5c3f2e58" is_pre_election: true
I20260809 09:33:05.031803 32577 raft_consensus.cc:2454] T 9ea69a45821c40099c23e6d5323a500b P 2a843b4644cd4b2389cf749e5c3f2e58 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b44b10e0ca7c4c9e89325813a36288d7 in term 0.
I20260809 09:33:05.031738 519 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ea69a45821c40099c23e6d5323a500b" candidate_uuid: "b44b10e0ca7c4c9e89325813a36288d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2313d46fd7f84023b99b769f25a2bd37" is_pre_election: true
I20260809 09:33:05.031961 519 raft_consensus.cc:2454] T 9ea69a45821c40099c23e6d5323a500b P 2313d46fd7f84023b99b769f25a2bd37 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b44b10e0ca7c4c9e89325813a36288d7 in term 0.
I20260809 09:33:05.032166 32685 leader_election.cc:302] T 9ea69a45821c40099c23e6d5323a500b P b44b10e0ca7c4c9e89325813a36288d7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a843b4644cd4b2389cf749e5c3f2e58, b44b10e0ca7c4c9e89325813a36288d7; no voters:
I20260809 09:33:05.032406 667 raft_consensus.cc:2781] T 9ea69a45821c40099c23e6d5323a500b P b44b10e0ca7c4c9e89325813a36288d7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 09:33:05.032495 667 raft_consensus.cc:482] T 9ea69a45821c40099c23e6d5323a500b P b44b10e0ca7c4c9e89325813a36288d7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 09:33:05.032554 667 raft_consensus.cc:3037] T 9ea69a45821c40099c23e6d5323a500b P b44b10e0ca7c4c9e89325813a36288d7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 09:33:05.033749 667 raft_consensus.cc:504] T 9ea69a45821c40099c23e6d5323a500b P b44b10e0ca7c4c9e89325813a36288d7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a843b4644cd4b2389cf749e5c3f2e58" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 40235 } } peers { permanent_uuid: "b44b10e0ca7c4c9e89325813a36288d7" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 33845 } } peers { permanent_uuid: "2313d46fd7f84023b99b769f25a2bd37" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 40493 } }
I20260809 09:33:05.034130 667 leader_election.cc:288] T 9ea69a45821c40099c23e6d5323a500b P b44b10e0ca7c4c9e89325813a36288d7 [CANDIDATE]: Term 1 election: Requested vote from peers 2a843b4644cd4b2389cf749e5c3f2e58 (127.20.125.129:40235), 2313d46fd7f84023b99b769f25a2bd37 (127.20.125.131:40493)
I20260809 09:33:05.034580 32577 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ea69a45821c40099c23e6d5323a500b" candidate_uuid: "b44b10e0ca7c4c9e89325813a36288d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a843b4644cd4b2389cf749e5c3f2e58"
I20260809 09:33:05.034600 519 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ea69a45821c40099c23e6d5323a500b" candidate_uuid: "b44b10e0ca7c4c9e89325813a36288d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2313d46fd7f84023b99b769f25a2bd37"
I20260809 09:33:05.034751 519 raft_consensus.cc:3037] T 9ea69a45821c40099c23e6d5323a500b P 2313d46fd7f84023b99b769f25a2bd37 [term 0 FOLLOWER]: Advancing to term 1
I20260809 09:33:05.034737 32577 raft_consensus.cc:3037] T 9ea69a45821c40099c23e6d5323a500b P 2a843b4644cd4b2389cf749e5c3f2e58 [term 0 FOLLOWER]: Advancing to term 1
I20260809 09:33:05.035807 519 raft_consensus.cc:2454] T 9ea69a45821c40099c23e6d5323a500b P 2313d46fd7f84023b99b769f25a2bd37 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b44b10e0ca7c4c9e89325813a36288d7 in term 1.
I20260809 09:33:05.035974 32577 raft_consensus.cc:2454] T 9ea69a45821c40099c23e6d5323a500b P 2a843b4644cd4b2389cf749e5c3f2e58 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b44b10e0ca7c4c9e89325813a36288d7 in term 1.
I20260809 09:33:05.036163 32692 leader_election.cc:302] T 9ea69a45821c40099c23e6d5323a500b P b44b10e0ca7c4c9e89325813a36288d7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2313d46fd7f84023b99b769f25a2bd37, b44b10e0ca7c4c9e89325813a36288d7; no voters:
I20260809 09:33:05.036366 667 raft_consensus.cc:2781] T 9ea69a45821c40099c23e6d5323a500b P b44b10e0ca7c4c9e89325813a36288d7 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 09:33:05.036482 667 raft_consensus.cc:686] T 9ea69a45821c40099c23e6d5323a500b P b44b10e0ca7c4c9e89325813a36288d7 [term 1 LEADER]: Becoming Leader. State: Replica: b44b10e0ca7c4c9e89325813a36288d7, State: Running, Role: LEADER
I20260809 09:33:05.036652 667 consensus_queue.cc:237] T 9ea69a45821c40099c23e6d5323a500b P b44b10e0ca7c4c9e89325813a36288d7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a843b4644cd4b2389cf749e5c3f2e58" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 40235 } } peers { permanent_uuid: "b44b10e0ca7c4c9e89325813a36288d7" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 33845 } } peers { permanent_uuid: "2313d46fd7f84023b99b769f25a2bd37" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 40493 } }
I20260809 09:33:05.038363 32340 catalog_manager.cc:5495] T 9ea69a45821c40099c23e6d5323a500b P b44b10e0ca7c4c9e89325813a36288d7 reported cstate change: term changed from 0 to 1, leader changed from <none> to b44b10e0ca7c4c9e89325813a36288d7 (127.20.125.130). New cstate: current_term: 1 leader_uuid: "b44b10e0ca7c4c9e89325813a36288d7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a843b4644cd4b2389cf749e5c3f2e58" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 40235 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b44b10e0ca7c4c9e89325813a36288d7" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 33845 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2313d46fd7f84023b99b769f25a2bd37" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 40493 } health_report { overall_health: UNKNOWN } } }
Aug 09 09:33:05 dist-test-slave-7kzw krb5kdc[32272](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786267983, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.125.190@KRBTEST.COM for oryx/127.20.125.190@KRBTEST.COM
I20260809 09:33:05.081238 32409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:33:05.072434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39930 (local address 127.20.125.190:43265)
0809 09:33:05.072616 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0809 09:33:05.072624 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 09:33:05.072815 (+ 191us) server_negotiation.cc:407] Connection header received
0809 09:33:05.072930 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:33:05.072932 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:33:05.072975 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:33:05.073057 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:33:05.073791 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:33:05.074309 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:33:05.075142 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:33:05.075355 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:33:05.075434 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:33:05.077940 (+ 2506us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:33:05.077943 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:33:05.077945 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:33:05.077965 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:33:05.079458 (+ 1493us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:33:05.079938 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:33:05.079943 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:33:05.079945 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:33:05.080022 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:33:05.080303 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:33:05.080307 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:33:05.080309 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:33:05.080481 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:33:05.080588 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0809 09:33:05.080890 (+ 302us) server_negotiation.cc:299] Negotiation successful
0809 09:33:05.081092 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":98,"server-negotiator.queue_time_us":75}
I20260809 09:33:05.081239 690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:33:05.072362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:43265 (local address 127.0.0.1:39930)
0809 09:33:05.072699 (+ 337us) negotiation.cc:107] Waiting for socket to connect
0809 09:33:05.072718 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 09:33:05.072842 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:33:05.073101 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:33:05.073103 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:33:05.073562 (+ 459us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:33:05.073663 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:33:05.073669 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:33:05.074412 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:33:05.074414 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:33:05.075032 (+ 618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:33:05.075041 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:33:05.075461 (+ 420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:33:05.075464 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:33:05.075526 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:33:05.076097 (+ 571us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 09:33:05.076111 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:33:05.077792 (+ 1681us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 09:33:05.079571 (+ 1779us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:33:05.079584 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:33:05.079591 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:33:05.079837 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:33:05.080113 (+ 276us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:33:05.080117 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:33:05.080119 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:33:05.080213 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:33:05.080589 (+ 376us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 09:33:05.080597 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 09:33:05.080715 (+ 118us) client_negotiation.cc:769] Sending connection context
0809 09:33:05.080844 (+ 129us) client_negotiation.cc:240] Negotiation successful
0809 09:33:05.081029 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":244,"thread_start_us":116,"threads_started":1}
I20260809 09:33:05.085678 32340 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.20.125.190@KRBTEST.COM'} at 127.0.0.1:39930:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260809 09:33:05.086374 32340 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 09:33:05.091595 32762 tablet_service.cc:1467] Processing CreateTablet for tablet d225229b02ae4123ba80e9e5e1b5c828 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=636a30114abe4cd9bfdb84ae757df29c]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 09:33:05.091627 32545 tablet_service.cc:1467] Processing CreateTablet for tablet d225229b02ae4123ba80e9e5e1b5c828 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=636a30114abe4cd9bfdb84ae757df29c]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 09:33:05.091681 487 tablet_service.cc:1467] Processing CreateTablet for tablet d225229b02ae4123ba80e9e5e1b5c828 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=636a30114abe4cd9bfdb84ae757df29c]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 09:33:05.092031 32762 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d225229b02ae4123ba80e9e5e1b5c828. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 09:33:05.092031 32545 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d225229b02ae4123ba80e9e5e1b5c828. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 09:33:05.092099 487 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d225229b02ae4123ba80e9e5e1b5c828. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 09:33:05.094856 631 tablet_bootstrap.cc:492] T d225229b02ae4123ba80e9e5e1b5c828 P 2313d46fd7f84023b99b769f25a2bd37: Bootstrap starting.
I20260809 09:33:05.094877 630 tablet_bootstrap.cc:492] T d225229b02ae4123ba80e9e5e1b5c828 P b44b10e0ca7c4c9e89325813a36288d7: Bootstrap starting.
I20260809 09:33:05.094902 632 tablet_bootstrap.cc:492] T d225229b02ae4123ba80e9e5e1b5c828 P 2a843b4644cd4b2389cf749e5c3f2e58: Bootstrap starting.
I20260809 09:33:05.095824 631 tablet_bootstrap.cc:654] T d225229b02ae4123ba80e9e5e1b5c828 P 2313d46fd7f84023b99b769f25a2bd37: Neither blocks nor log segments found. Creating new log.
I20260809 09:33:05.095824 632 tablet_bootstrap.cc:654] T d225229b02ae4123ba80e9e5e1b5c828 P 2a843b4644cd4b2389cf749e5c3f2e58: Neither blocks nor log segments found. Creating new log.
I20260809 09:33:05.095824 630 tablet_bootstrap.cc:654] T d225229b02ae4123ba80e9e5e1b5c828 P b44b10e0ca7c4c9e89325813a36288d7: Neither blocks nor log segments found. Creating new log.
I20260809 09:33:05.097288 631 tablet_bootstrap.cc:492] T d225229b02ae4123ba80e9e5e1b5c828 P 2313d46fd7f84023b99b769f25a2bd37: No bootstrap required, opened a new log
I20260809 09:33:05.097396 631 ts_tablet_manager.cc:1397] T d225229b02ae4123ba80e9e5e1b5c828 P 2313d46fd7f84023b99b769f25a2bd37: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260809 09:33:05.097553 630 tablet_bootstrap.cc:492] T d225229b02ae4123ba80e9e5e1b5c828 P b44b10e0ca7c4c9e89325813a36288d7: No bootstrap required, opened a new log
I20260809 09:33:05.097647 630 ts_tablet_manager.cc:1397] T d225229b02ae4123ba80e9e5e1b5c828 P b44b10e0ca7c4c9e89325813a36288d7: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260809 09:33:05.097757 632 tablet_bootstrap.cc:492] T d225229b02ae4123ba80e9e5e1b5c828 P 2a843b4644cd4b2389cf749e5c3f2e58: No bootstrap required, opened a new log
I20260809 09:33:05.097841 632 ts_tablet_manager.cc:1397] T d225229b02ae4123ba80e9e5e1b5c828 P 2a843b4644cd4b2389cf749e5c3f2e58: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 09:33:05.097931 631 raft_consensus.cc:348] T d225229b02ae4123ba80e9e5e1b5c828 P 2313d46fd7f84023b99b769f25a2bd37 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a843b4644cd4b2389cf749e5c3f2e58" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 40235 } } peers { permanent_uuid: "b44b10e0ca7c4c9e89325813a36288d7" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 33845 } } peers { permanent_uuid: "2313d46fd7f84023b99b769f25a2bd37" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 40493 } }
I20260809 09:33:05.098076 631 raft_consensus.cc:374] T d225229b02ae4123ba80e9e5e1b5c828 P 2313d46fd7f84023b99b769f25a2bd37 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 09:33:05.098117 631 raft_consensus.cc:729] T d225229b02ae4123ba80e9e5e1b5c828 P 2313d46fd7f84023b99b769f25a2bd37 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2313d46fd7f84023b99b769f25a2bd37, State: Initialized, Role: FOLLOWER
I20260809 09:33:05.098078 630 raft_consensus.cc:348] T d225229b02ae4123ba80e9e5e1b5c828 P b44b10e0ca7c4c9e89325813a36288d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a843b4644cd4b2389cf749e5c3f2e58" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 40235 } } peers { permanent_uuid: "b44b10e0ca7c4c9e89325813a36288d7" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 33845 } } peers { permanent_uuid: "2313d46fd7f84023b99b769f25a2bd37" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 40493 } }
I20260809 09:33:05.098210 630 raft_consensus.cc:374] T d225229b02ae4123ba80e9e5e1b5c828 P b44b10e0ca7c4c9e89325813a36288d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 09:33:05.098258 630 raft_consensus.cc:729] T d225229b02ae4123ba80e9e5e1b5c828 P b44b10e0ca7c4c9e89325813a36288d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b44b10e0ca7c4c9e89325813a36288d7, State: Initialized, Role: FOLLOWER
I20260809 09:33:05.098260 631 consensus_queue.cc:260] T d225229b02ae4123ba80e9e5e1b5c828 P 2313d46fd7f84023b99b769f25a2bd37 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a843b4644cd4b2389cf749e5c3f2e58" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 40235 } } peers { permanent_uuid: "b44b10e0ca7c4c9e89325813a36288d7" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 33845 } } peers { permanent_uuid: "2313d46fd7f84023b99b769f25a2bd37" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 40493 } }
I20260809 09:33:05.098292 632 raft_consensus.cc:348] T d225229b02ae4123ba80e9e5e1b5c828 P 2a843b4644cd4b2389cf749e5c3f2e58 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a843b4644cd4b2389cf749e5c3f2e58" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 40235 } } peers { permanent_uuid: "b44b10e0ca7c4c9e89325813a36288d7" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 33845 } } peers { permanent_uuid: "2313d46fd7f84023b99b769f25a2bd37" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 40493 } }
I20260809 09:33:05.098394 632 raft_consensus.cc:374] T d225229b02ae4123ba80e9e5e1b5c828 P 2a843b4644cd4b2389cf749e5c3f2e58 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 09:33:05.098436 632 raft_consensus.cc:729] T d225229b02ae4123ba80e9e5e1b5c828 P 2a843b4644cd4b2389cf749e5c3f2e58 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a843b4644cd4b2389cf749e5c3f2e58, State: Initialized, Role: FOLLOWER
I20260809 09:33:05.098397 630 consensus_queue.cc:260] T d225229b02ae4123ba80e9e5e1b5c828 P b44b10e0ca7c4c9e89325813a36288d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a843b4644cd4b2389cf749e5c3f2e58" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 40235 } } peers { permanent_uuid: "b44b10e0ca7c4c9e89325813a36288d7" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 33845 } } peers { permanent_uuid: "2313d46fd7f84023b99b769f25a2bd37" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 40493 } }
I20260809 09:33:05.098587 632 consensus_queue.cc:260] T d225229b02ae4123ba80e9e5e1b5c828 P 2a843b4644cd4b2389cf749e5c3f2e58 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a843b4644cd4b2389cf749e5c3f2e58" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 40235 } } peers { permanent_uuid: "b44b10e0ca7c4c9e89325813a36288d7" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 33845 } } peers { permanent_uuid: "2313d46fd7f84023b99b769f25a2bd37" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 40493 } }
I20260809 09:33:05.098592 646 tablet_replica.cc:439] T d225229b02ae4123ba80e9e5e1b5c828 P 2313d46fd7f84023b99b769f25a2bd37: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a843b4644cd4b2389cf749e5c3f2e58" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 40235 } } peers { permanent_uuid: "b44b10e0ca7c4c9e89325813a36288d7" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 33845 } } peers { permanent_uuid: "2313d46fd7f84023b99b769f25a2bd37" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 40493 } } }
I20260809 09:33:05.098749 646 tablet_replica.cc:442] T d225229b02ae4123ba80e9e5e1b5c828 P 2313d46fd7f84023b99b769f25a2bd37: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 09:33:05.098701 667 tablet_replica.cc:439] T d225229b02ae4123ba80e9e5e1b5c828 P b44b10e0ca7c4c9e89325813a36288d7: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a843b4644cd4b2389cf749e5c3f2e58" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 40235 } } peers { permanent_uuid: "b44b10e0ca7c4c9e89325813a36288d7" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 33845 } } peers { permanent_uuid: "2313d46fd7f84023b99b769f25a2bd37" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 40493 } } }
I20260809 09:33:05.098762 630 ts_tablet_manager.cc:1428] T d225229b02ae4123ba80e9e5e1b5c828 P b44b10e0ca7c4c9e89325813a36288d7: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260809 09:33:05.098819 667 tablet_replica.cc:442] T d225229b02ae4123ba80e9e5e1b5c828 P b44b10e0ca7c4c9e89325813a36288d7: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 09:33:05.098884 632 ts_tablet_manager.cc:1428] T d225229b02ae4123ba80e9e5e1b5c828 P 2a843b4644cd4b2389cf749e5c3f2e58: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260809 09:33:05.099108 649 tablet_replica.cc:439] T d225229b02ae4123ba80e9e5e1b5c828 P 2a843b4644cd4b2389cf749e5c3f2e58: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a843b4644cd4b2389cf749e5c3f2e58" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 40235 } } peers { permanent_uuid: "b44b10e0ca7c4c9e89325813a36288d7" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 33845 } } peers { permanent_uuid: "2313d46fd7f84023b99b769f25a2bd37" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 40493 } } }
I20260809 09:33:05.099246 649 tablet_replica.cc:442] T d225229b02ae4123ba80e9e5e1b5c828 P 2a843b4644cd4b2389cf749e5c3f2e58: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 09:33:05.099886 631 ts_tablet_manager.cc:1428] T d225229b02ae4123ba80e9e5e1b5c828 P 2313d46fd7f84023b99b769f25a2bd37: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260809 09:33:05.234081 646 raft_consensus.cc:482] T d225229b02ae4123ba80e9e5e1b5c828 P 2313d46fd7f84023b99b769f25a2bd37 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 09:33:05.234248 646 raft_consensus.cc:504] T d225229b02ae4123ba80e9e5e1b5c828 P 2313d46fd7f84023b99b769f25a2bd37 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a843b4644cd4b2389cf749e5c3f2e58" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 40235 } } peers { permanent_uuid: "b44b10e0ca7c4c9e89325813a36288d7" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 33845 } } peers { permanent_uuid: "2313d46fd7f84023b99b769f25a2bd37" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 40493 } }
I20260809 09:33:05.235519 646 leader_election.cc:288] T d225229b02ae4123ba80e9e5e1b5c828 P 2313d46fd7f84023b99b769f25a2bd37 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2a843b4644cd4b2389cf749e5c3f2e58 (127.20.125.129:40235), b44b10e0ca7c4c9e89325813a36288d7 (127.20.125.130:33845)
I20260809 09:33:05.240314 695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:33:05.236289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.130:33845 (local address 127.20.125.131:51341)
0809 09:33:05.236637 (+ 348us) negotiation.cc:107] Waiting for socket to connect
0809 09:33:05.236650 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 09:33:05.236747 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:33:05.237052 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:33:05.237054 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:33:05.237066 (+ 12us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 09:33:05.237143 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:33:05.237148 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:33:05.238062 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:33:05.238066 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:33:05.239036 (+ 970us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:33:05.239047 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:33:05.239850 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:33:05.239854 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:33:05.239962 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:33:05.239988 (+ 26us) client_negotiation.cc:769] Sending connection context
0809 09:33:05.240065 (+ 77us) client_negotiation.cc:240] Negotiation successful
0809 09:33:05.240135 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":284,"mutex_wait_us":40,"thread_start_us":80,"threads_started":1}
I20260809 09:33:05.240309 628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:33:05.236441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.131:51341 (local address 127.20.125.130:33845)
0809 09:33:05.236590 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0809 09:33:05.236594 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 09:33:05.236719 (+ 125us) server_negotiation.cc:407] Connection header received
0809 09:33:05.236889 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:33:05.236893 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:33:05.236948 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:33:05.237036 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 09:33:05.237247 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:33:05.237958 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:33:05.239167 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:33:05.239728 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:33:05.239851 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:33:05.239907 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0809 09:33:05.240104 (+ 197us) server_negotiation.cc:299] Negotiation successful
0809 09:33:05.240176 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260809 09:33:05.240715 622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:33:05.235892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.131:41871 (local address 127.20.125.129:40235)
0809 09:33:05.236044 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0809 09:33:05.236049 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 09:33:05.236518 (+ 469us) server_negotiation.cc:407] Connection header received
0809 09:33:05.236581 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:33:05.236584 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:33:05.236635 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:33:05.236733 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 09:33:05.237059 (+ 326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:33:05.237878 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:33:05.239433 (+ 1555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:33:05.240003 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:33:05.240135 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:33:05.240190 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0809 09:33:05.240523 (+ 333us) server_negotiation.cc:299] Negotiation successful
0809 09:33:05.240599 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 09:33:05.241120 694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:33:05.235806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.129:40235 (local address 127.20.125.131:41871)
0809 09:33:05.236123 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0809 09:33:05.236138 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 09:33:05.236294 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:33:05.236786 (+ 492us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:33:05.236789 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:33:05.236809 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 09:33:05.236926 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:33:05.236932 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:33:05.237992 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:33:05.237995 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:33:05.239299 (+ 1304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:33:05.239311 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:33:05.240253 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:33:05.240257 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:33:05.240364 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:33:05.240391 (+ 27us) client_negotiation.cc:769] Sending connection context
0809 09:33:05.240911 (+ 520us) client_negotiation.cc:240] Negotiation successful
0809 09:33:05.240989 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":221,"mutex_wait_us":2,"thread_start_us":105,"threads_started":1}
I20260809 09:33:05.241211 323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d225229b02ae4123ba80e9e5e1b5c828" candidate_uuid: "2313d46fd7f84023b99b769f25a2bd37" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b44b10e0ca7c4c9e89325813a36288d7" is_pre_election: true
I20260809 09:33:05.241474 323 raft_consensus.cc:2454] T d225229b02ae4123ba80e9e5e1b5c828 P b44b10e0ca7c4c9e89325813a36288d7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2313d46fd7f84023b99b769f25a2bd37 in term 0.
I20260809 09:33:05.241740 32577 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d225229b02ae4123ba80e9e5e1b5c828" candidate_uuid: "2313d46fd7f84023b99b769f25a2bd37" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a843b4644cd4b2389cf749e5c3f2e58" is_pre_election: true
I20260809 09:33:05.241911 32577 raft_consensus.cc:2454] T d225229b02ae4123ba80e9e5e1b5c828 P 2a843b4644cd4b2389cf749e5c3f2e58 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2313d46fd7f84023b99b769f25a2bd37 in term 0.
I20260809 09:33:05.241876 420 leader_election.cc:302] T d225229b02ae4123ba80e9e5e1b5c828 P 2313d46fd7f84023b99b769f25a2bd37 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2313d46fd7f84023b99b769f25a2bd37, b44b10e0ca7c4c9e89325813a36288d7; no voters:
I20260809 09:33:05.242136 646 raft_consensus.cc:2781] T d225229b02ae4123ba80e9e5e1b5c828 P 2313d46fd7f84023b99b769f25a2bd37 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 09:33:05.242200 646 raft_consensus.cc:482] T d225229b02ae4123ba80e9e5e1b5c828 P 2313d46fd7f84023b99b769f25a2bd37 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 09:33:05.242251 646 raft_consensus.cc:3037] T d225229b02ae4123ba80e9e5e1b5c828 P 2313d46fd7f84023b99b769f25a2bd37 [term 0 FOLLOWER]: Advancing to term 1
I20260809 09:33:05.243340 646 raft_consensus.cc:504] T d225229b02ae4123ba80e9e5e1b5c828 P 2313d46fd7f84023b99b769f25a2bd37 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a843b4644cd4b2389cf749e5c3f2e58" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 40235 } } peers { permanent_uuid: "b44b10e0ca7c4c9e89325813a36288d7" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 33845 } } peers { permanent_uuid: "2313d46fd7f84023b99b769f25a2bd37" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 40493 } }
I20260809 09:33:05.243708 646 leader_election.cc:288] T d225229b02ae4123ba80e9e5e1b5c828 P 2313d46fd7f84023b99b769f25a2bd37 [CANDIDATE]: Term 1 election: Requested vote from peers 2a843b4644cd4b2389cf749e5c3f2e58 (127.20.125.129:40235), b44b10e0ca7c4c9e89325813a36288d7 (127.20.125.130:33845)
I20260809 09:33:05.244149 32577 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d225229b02ae4123ba80e9e5e1b5c828" candidate_uuid: "2313d46fd7f84023b99b769f25a2bd37" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a843b4644cd4b2389cf749e5c3f2e58"
I20260809 09:33:05.244149 323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d225229b02ae4123ba80e9e5e1b5c828" candidate_uuid: "2313d46fd7f84023b99b769f25a2bd37" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b44b10e0ca7c4c9e89325813a36288d7"
I20260809 09:33:05.244280 32577 raft_consensus.cc:3037] T d225229b02ae4123ba80e9e5e1b5c828 P 2a843b4644cd4b2389cf749e5c3f2e58 [term 0 FOLLOWER]: Advancing to term 1
I20260809 09:33:05.244280 323 raft_consensus.cc:3037] T d225229b02ae4123ba80e9e5e1b5c828 P b44b10e0ca7c4c9e89325813a36288d7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 09:33:05.245409 32577 raft_consensus.cc:2454] T d225229b02ae4123ba80e9e5e1b5c828 P 2a843b4644cd4b2389cf749e5c3f2e58 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2313d46fd7f84023b99b769f25a2bd37 in term 1.
I20260809 09:33:05.245450 323 raft_consensus.cc:2454] T d225229b02ae4123ba80e9e5e1b5c828 P b44b10e0ca7c4c9e89325813a36288d7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2313d46fd7f84023b99b769f25a2bd37 in term 1.
I20260809 09:33:05.245782 420 leader_election.cc:302] T d225229b02ae4123ba80e9e5e1b5c828 P 2313d46fd7f84023b99b769f25a2bd37 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2313d46fd7f84023b99b769f25a2bd37, b44b10e0ca7c4c9e89325813a36288d7; no voters:
I20260809 09:33:05.246034 646 raft_consensus.cc:2781] T d225229b02ae4123ba80e9e5e1b5c828 P 2313d46fd7f84023b99b769f25a2bd37 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 09:33:05.246239 646 raft_consensus.cc:686] T d225229b02ae4123ba80e9e5e1b5c828 P 2313d46fd7f84023b99b769f25a2bd37 [term 1 LEADER]: Becoming Leader. State: Replica: 2313d46fd7f84023b99b769f25a2bd37, State: Running, Role: LEADER
I20260809 09:33:05.246541 646 consensus_queue.cc:237] T d225229b02ae4123ba80e9e5e1b5c828 P 2313d46fd7f84023b99b769f25a2bd37 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a843b4644cd4b2389cf749e5c3f2e58" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 40235 } } peers { permanent_uuid: "b44b10e0ca7c4c9e89325813a36288d7" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 33845 } } peers { permanent_uuid: "2313d46fd7f84023b99b769f25a2bd37" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 40493 } }
I20260809 09:33:05.247936 696 tablet_replica.cc:439] T d225229b02ae4123ba80e9e5e1b5c828 P 2313d46fd7f84023b99b769f25a2bd37: TxnStatusTablet state changed. Reason: New leader 2313d46fd7f84023b99b769f25a2bd37. Latest consensus state: current_term: 1 leader_uuid: "2313d46fd7f84023b99b769f25a2bd37" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a843b4644cd4b2389cf749e5c3f2e58" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 40235 } } peers { permanent_uuid: "b44b10e0ca7c4c9e89325813a36288d7" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 33845 } } peers { permanent_uuid: "2313d46fd7f84023b99b769f25a2bd37" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 40493 } } }
I20260809 09:33:05.248052 696 tablet_replica.cc:442] T d225229b02ae4123ba80e9e5e1b5c828 P 2313d46fd7f84023b99b769f25a2bd37: This TxnStatusTablet replica's current role is: LEADER
I20260809 09:33:05.248781 698 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260809 09:33:05.248945 698 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260809 09:33:05.249277 32334 catalog_manager.cc:5495] T d225229b02ae4123ba80e9e5e1b5c828 P 2313d46fd7f84023b99b769f25a2bd37 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2313d46fd7f84023b99b769f25a2bd37 (127.20.125.131). New cstate: current_term: 1 leader_uuid: "2313d46fd7f84023b99b769f25a2bd37" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a843b4644cd4b2389cf749e5c3f2e58" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 40235 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b44b10e0ca7c4c9e89325813a36288d7" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 33845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2313d46fd7f84023b99b769f25a2bd37" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 40493 } health_report { overall_health: HEALTHY } } }
I20260809 09:33:05.294785 690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:33:05.290175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.131:40493 (local address 127.0.0.1:38154)
0809 09:33:05.290321 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0809 09:33:05.290339 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 09:33:05.290448 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:33:05.290978 (+ 530us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:33:05.290983 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:33:05.290996 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 09:33:05.291126 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:33:05.291136 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:33:05.292256 (+ 1120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:33:05.292263 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:33:05.292897 (+ 634us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:33:05.292907 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:33:05.293517 (+ 610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:33:05.293520 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:33:05.293595 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:33:05.293651 (+ 56us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 09:33:05.294439 (+ 788us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 09:33:05.294454 (+ 15us) client_negotiation.cc:769] Sending connection context
0809 09:33:05.294582 (+ 128us) client_negotiation.cc:240] Negotiation successful
0809 09:33:05.294664 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260809 09:33:05.294822 625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:33:05.290283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38154 (local address 127.20.125.131:40493)
0809 09:33:05.290414 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0809 09:33:05.290419 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 09:33:05.290436 (+ 17us) server_negotiation.cc:407] Connection header received
0809 09:33:05.290651 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:33:05.290655 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:33:05.290783 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:33:05.290897 (+ 114us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 09:33:05.291265 (+ 368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:33:05.292097 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:33:05.293020 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:33:05.293379 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:33:05.293493 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:33:05.293801 (+ 308us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 09:33:05.294334 (+ 533us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 09:33:05.294448 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0809 09:33:05.294615 (+ 167us) server_negotiation.cc:299] Negotiation successful
0809 09:33:05.294676 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 09:33:05.297976 32577 raft_consensus.cc:1261] T d225229b02ae4123ba80e9e5e1b5c828 P 2a843b4644cd4b2389cf749e5c3f2e58 [term 1 FOLLOWER]: Refusing update from remote peer 2313d46fd7f84023b99b769f25a2bd37: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 09:33:05.298213 323 raft_consensus.cc:1261] T d225229b02ae4123ba80e9e5e1b5c828 P b44b10e0ca7c4c9e89325813a36288d7 [term 1 FOLLOWER]: Refusing update from remote peer 2313d46fd7f84023b99b769f25a2bd37: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 09:33:05.298542 696 consensus_queue.cc:1035] T d225229b02ae4123ba80e9e5e1b5c828 P 2313d46fd7f84023b99b769f25a2bd37 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2a843b4644cd4b2389cf749e5c3f2e58" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 40235 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 09:33:05.298820 646 consensus_queue.cc:1035] T d225229b02ae4123ba80e9e5e1b5c828 P 2313d46fd7f84023b99b769f25a2bd37 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b44b10e0ca7c4c9e89325813a36288d7" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 33845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 09:33:05.300164 649 tablet_replica.cc:439] T d225229b02ae4123ba80e9e5e1b5c828 P 2a843b4644cd4b2389cf749e5c3f2e58: TxnStatusTablet state changed. Reason: New leader 2313d46fd7f84023b99b769f25a2bd37. Latest consensus state: current_term: 1 leader_uuid: "2313d46fd7f84023b99b769f25a2bd37" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a843b4644cd4b2389cf749e5c3f2e58" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 40235 } } peers { permanent_uuid: "b44b10e0ca7c4c9e89325813a36288d7" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 33845 } } peers { permanent_uuid: "2313d46fd7f84023b99b769f25a2bd37" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 40493 } } }
I20260809 09:33:05.300288 649 tablet_replica.cc:442] T d225229b02ae4123ba80e9e5e1b5c828 P 2a843b4644cd4b2389cf749e5c3f2e58: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 09:33:05.300508 667 tablet_replica.cc:439] T d225229b02ae4123ba80e9e5e1b5c828 P b44b10e0ca7c4c9e89325813a36288d7: TxnStatusTablet state changed. Reason: New leader 2313d46fd7f84023b99b769f25a2bd37. Latest consensus state: current_term: 1 leader_uuid: "2313d46fd7f84023b99b769f25a2bd37" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a843b4644cd4b2389cf749e5c3f2e58" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 40235 } } peers { permanent_uuid: "b44b10e0ca7c4c9e89325813a36288d7" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 33845 } } peers { permanent_uuid: "2313d46fd7f84023b99b769f25a2bd37" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 40493 } } }
I20260809 09:33:05.300611 667 tablet_replica.cc:442] T d225229b02ae4123ba80e9e5e1b5c828 P b44b10e0ca7c4c9e89325813a36288d7: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 09:33:05.301792 696 tablet_replica.cc:439] T d225229b02ae4123ba80e9e5e1b5c828 P 2313d46fd7f84023b99b769f25a2bd37: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2313d46fd7f84023b99b769f25a2bd37" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a843b4644cd4b2389cf749e5c3f2e58" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 40235 } } peers { permanent_uuid: "b44b10e0ca7c4c9e89325813a36288d7" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 33845 } } peers { permanent_uuid: "2313d46fd7f84023b99b769f25a2bd37" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 40493 } } }
I20260809 09:33:05.301923 696 tablet_replica.cc:442] T d225229b02ae4123ba80e9e5e1b5c828 P 2313d46fd7f84023b99b769f25a2bd37: This TxnStatusTablet replica's current role is: LEADER
I20260809 09:33:05.304531 646 tablet_replica.cc:439] T d225229b02ae4123ba80e9e5e1b5c828 P 2313d46fd7f84023b99b769f25a2bd37: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2313d46fd7f84023b99b769f25a2bd37" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a843b4644cd4b2389cf749e5c3f2e58" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 40235 } } peers { permanent_uuid: "b44b10e0ca7c4c9e89325813a36288d7" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 33845 } } peers { permanent_uuid: "2313d46fd7f84023b99b769f25a2bd37" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 40493 } } }
I20260809 09:33:05.304651 646 tablet_replica.cc:442] T d225229b02ae4123ba80e9e5e1b5c828 P 2313d46fd7f84023b99b769f25a2bd37: This TxnStatusTablet replica's current role is: LEADER
I20260809 09:33:05.305984 667 tablet_replica.cc:439] T d225229b02ae4123ba80e9e5e1b5c828 P b44b10e0ca7c4c9e89325813a36288d7: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2313d46fd7f84023b99b769f25a2bd37" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a843b4644cd4b2389cf749e5c3f2e58" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 40235 } } peers { permanent_uuid: "b44b10e0ca7c4c9e89325813a36288d7" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 33845 } } peers { permanent_uuid: "2313d46fd7f84023b99b769f25a2bd37" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 40493 } } }
I20260809 09:33:05.306106 667 tablet_replica.cc:442] T d225229b02ae4123ba80e9e5e1b5c828 P b44b10e0ca7c4c9e89325813a36288d7: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 09:33:05.306911 661 mvcc.cc:205] Tried to move back new op lower bound from 7316553667774828544 to 7316553667572822016. Current Snapshot: MvccSnapshot[applied={T|T < 7316553667774828544}]
I20260809 09:33:05.307514 649 tablet_replica.cc:439] T d225229b02ae4123ba80e9e5e1b5c828 P 2a843b4644cd4b2389cf749e5c3f2e58: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2313d46fd7f84023b99b769f25a2bd37" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a843b4644cd4b2389cf749e5c3f2e58" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 40235 } } peers { permanent_uuid: "b44b10e0ca7c4c9e89325813a36288d7" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 33845 } } peers { permanent_uuid: "2313d46fd7f84023b99b769f25a2bd37" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 40493 } } }
I20260809 09:33:05.307634 649 tablet_replica.cc:442] T d225229b02ae4123ba80e9e5e1b5c828 P 2a843b4644cd4b2389cf749e5c3f2e58: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 09:33:05.317821 625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:33:05.314165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.130:58807 (local address 127.20.125.131:40493)
0809 09:33:05.314305 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0809 09:33:05.314309 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 09:33:05.314557 (+ 248us) server_negotiation.cc:407] Connection header received
0809 09:33:05.314616 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:33:05.314618 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:33:05.314661 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:33:05.314733 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 09:33:05.315001 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:33:05.315491 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:33:05.316728 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:33:05.316940 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:33:05.317022 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:33:05.317363 (+ 341us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 09:33:05.317432 (+ 69us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 09:33:05.317491 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0809 09:33:05.317615 (+ 124us) server_negotiation.cc:299] Negotiation successful
0809 09:33:05.317659 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260809 09:33:05.317842 709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:33:05.314061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.131:40493 (local address 127.20.125.130:58807)
0809 09:33:05.314380 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0809 09:33:05.314396 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 09:33:05.314525 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:33:05.314768 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:33:05.314771 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:33:05.314779 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 09:33:05.314891 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:33:05.314897 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:33:05.315600 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:33:05.315604 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:33:05.316593 (+ 989us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:33:05.316605 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:33:05.317053 (+ 448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:33:05.317057 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:33:05.317149 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:33:05.317202 (+ 53us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 09:33:05.317517 (+ 315us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 09:33:05.317525 (+ 8us) client_negotiation.cc:769] Sending connection context
0809 09:33:05.317591 (+ 66us) client_negotiation.cc:240] Negotiation successful
0809 09:33:05.317678 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":72,"threads_started":1}
I20260809 09:33:05.325073 32577 raft_consensus.cc:1261] T 9ea69a45821c40099c23e6d5323a500b P 2a843b4644cd4b2389cf749e5c3f2e58 [term 1 FOLLOWER]: Refusing update from remote peer b44b10e0ca7c4c9e89325813a36288d7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 09:33:05.325230 519 raft_consensus.cc:1261] T 9ea69a45821c40099c23e6d5323a500b P 2313d46fd7f84023b99b769f25a2bd37 [term 1 FOLLOWER]: Refusing update from remote peer b44b10e0ca7c4c9e89325813a36288d7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 09:33:05.325621 667 consensus_queue.cc:1035] T 9ea69a45821c40099c23e6d5323a500b P b44b10e0ca7c4c9e89325813a36288d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2a843b4644cd4b2389cf749e5c3f2e58" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 40235 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 09:33:05.325906 667 consensus_queue.cc:1035] T 9ea69a45821c40099c23e6d5323a500b P b44b10e0ca7c4c9e89325813a36288d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2313d46fd7f84023b99b769f25a2bd37" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 40493 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 09:33:05.348490 628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:33:05.343892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.131:50351 (local address 127.20.125.130:33845)
0809 09:33:05.343994 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0809 09:33:05.343999 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 09:33:05.344012 (+ 13us) server_negotiation.cc:407] Connection header received
0809 09:33:05.344055 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:33:05.344058 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:33:05.344107 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:33:05.344185 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 09:33:05.344764 (+ 579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:33:05.345542 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:33:05.346850 (+ 1308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:33:05.347177 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:33:05.347297 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:33:05.347855 (+ 558us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 09:33:05.347944 (+ 89us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 09:33:05.348015 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0809 09:33:05.348309 (+ 294us) server_negotiation.cc:299] Negotiation successful
0809 09:33:05.348363 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260809 09:33:05.348549 717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:33:05.343517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.130:33845 (local address 127.20.125.131:50351)
0809 09:33:05.343841 (+ 324us) negotiation.cc:107] Waiting for socket to connect
0809 09:33:05.343856 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 09:33:05.343953 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:33:05.344506 (+ 553us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:33:05.344509 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:33:05.344518 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 09:33:05.344618 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:33:05.344624 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:33:05.345684 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:33:05.345687 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:33:05.346724 (+ 1037us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:33:05.346738 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:33:05.347512 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:33:05.347515 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:33:05.347647 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:33:05.347699 (+ 52us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 09:33:05.348120 (+ 421us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 09:33:05.348131 (+ 11us) client_negotiation.cc:769] Sending connection context
0809 09:33:05.348235 (+ 104us) client_negotiation.cc:240] Negotiation successful
0809 09:33:05.348338 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":120,"threads_started":1}
I20260809 09:33:05.373680 32334 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39918:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 09:33:05.373872 32334 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39918:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 09:33:05.376369 32334 catalog_manager.cc:5782] T 00000000000000000000000000000000 P fdd08fcf074d4c108a4ea936f91b202d: Sending DeleteTablet for 3 replicas of tablet 9ea69a45821c40099c23e6d5323a500b
I20260809 09:33:05.376950 32545 tablet_service.cc:1514] Processing DeleteTablet for tablet 9ea69a45821c40099c23e6d5323a500b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 09:33:05 UTC) from {username='oryx', principal='oryx/127.20.125.190@KRBTEST.COM'} at 127.0.0.1:49832
I20260809 09:33:05.377002 32762 tablet_service.cc:1514] Processing DeleteTablet for tablet 9ea69a45821c40099c23e6d5323a500b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 09:33:05 UTC) from {username='oryx', principal='oryx/127.20.125.190@KRBTEST.COM'} at 127.0.0.1:56048
I20260809 09:33:05.377300 723 tablet_replica.cc:330] T 9ea69a45821c40099c23e6d5323a500b P b44b10e0ca7c4c9e89325813a36288d7: stopping tablet replica
I20260809 09:33:05.377310 720 tablet_replica.cc:330] T 9ea69a45821c40099c23e6d5323a500b P 2a843b4644cd4b2389cf749e5c3f2e58: stopping tablet replica
I20260809 09:33:05.377431 723 raft_consensus.cc:2229] T 9ea69a45821c40099c23e6d5323a500b P b44b10e0ca7c4c9e89325813a36288d7 [term 1 LEADER]: Raft consensus shutting down.
I20260809 09:33:05.377440 720 raft_consensus.cc:2229] T 9ea69a45821c40099c23e6d5323a500b P 2a843b4644cd4b2389cf749e5c3f2e58 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 09:33:05.377460 487 tablet_service.cc:1514] Processing DeleteTablet for tablet 9ea69a45821c40099c23e6d5323a500b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 09:33:05 UTC) from {username='oryx', principal='oryx/127.20.125.190@KRBTEST.COM'} at 127.0.0.1:38144
I20260809 09:33:05.377552 720 raft_consensus.cc:2258] T 9ea69a45821c40099c23e6d5323a500b P 2a843b4644cd4b2389cf749e5c3f2e58 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 09:33:05.377676 723 raft_consensus.cc:2258] T 9ea69a45821c40099c23e6d5323a500b P b44b10e0ca7c4c9e89325813a36288d7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 09:33:05.378341 724 tablet_replica.cc:330] T 9ea69a45821c40099c23e6d5323a500b P 2313d46fd7f84023b99b769f25a2bd37: stopping tablet replica
I20260809 09:33:05.378549 724 raft_consensus.cc:2229] T 9ea69a45821c40099c23e6d5323a500b P 2313d46fd7f84023b99b769f25a2bd37 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 09:33:05.378715 724 raft_consensus.cc:2258] T 9ea69a45821c40099c23e6d5323a500b P 2313d46fd7f84023b99b769f25a2bd37 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 09:33:05.379235 723 ts_tablet_manager.cc:1905] T 9ea69a45821c40099c23e6d5323a500b P b44b10e0ca7c4c9e89325813a36288d7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 09:33:05.379426 724 ts_tablet_manager.cc:1905] T 9ea69a45821c40099c23e6d5323a500b P 2313d46fd7f84023b99b769f25a2bd37: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 09:33:05.382256 720 ts_tablet_manager.cc:1905] T 9ea69a45821c40099c23e6d5323a500b P 2a843b4644cd4b2389cf749e5c3f2e58: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 09:33:05.382416 724 ts_tablet_manager.cc:1918] T 9ea69a45821c40099c23e6d5323a500b P 2313d46fd7f84023b99b769f25a2bd37: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 09:33:05.382561 723 ts_tablet_manager.cc:1918] T 9ea69a45821c40099c23e6d5323a500b P b44b10e0ca7c4c9e89325813a36288d7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 09:33:05.382586 724 log.cc:1196] T 9ea69a45821c40099c23e6d5323a500b P 2313d46fd7f84023b99b769f25a2bd37: Deleting WAL directory at /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786267941944493-20982-0/minicluster-data/ts-2/wal/wals/9ea69a45821c40099c23e6d5323a500b
I20260809 09:33:05.382654 723 log.cc:1196] T 9ea69a45821c40099c23e6d5323a500b P b44b10e0ca7c4c9e89325813a36288d7: Deleting WAL directory at /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786267941944493-20982-0/minicluster-data/ts-1/wal/wals/9ea69a45821c40099c23e6d5323a500b
I20260809 09:33:05.382977 723 ts_tablet_manager.cc:1939] T 9ea69a45821c40099c23e6d5323a500b P b44b10e0ca7c4c9e89325813a36288d7: Deleting consensus metadata
I20260809 09:33:05.383010 724 ts_tablet_manager.cc:1939] T 9ea69a45821c40099c23e6d5323a500b P 2313d46fd7f84023b99b769f25a2bd37: Deleting consensus metadata
I20260809 09:33:05.383721 32318 catalog_manager.cc:4841] TS 2313d46fd7f84023b99b769f25a2bd37 (127.20.125.131:40493): tablet 9ea69a45821c40099c23e6d5323a500b (table test-table [id=f3a12d6c177347ea83fcaf9d2804aa05]) successfully deleted
I20260809 09:33:05.383721 32316 catalog_manager.cc:4841] TS b44b10e0ca7c4c9e89325813a36288d7 (127.20.125.130:33845): tablet 9ea69a45821c40099c23e6d5323a500b (table test-table [id=f3a12d6c177347ea83fcaf9d2804aa05]) successfully deleted
I20260809 09:33:05.385521 720 ts_tablet_manager.cc:1918] T 9ea69a45821c40099c23e6d5323a500b P 2a843b4644cd4b2389cf749e5c3f2e58: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 09:33:05.385615 720 log.cc:1196] T 9ea69a45821c40099c23e6d5323a500b P 2a843b4644cd4b2389cf749e5c3f2e58: Deleting WAL directory at /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786267941944493-20982-0/minicluster-data/ts-0/wal/wals/9ea69a45821c40099c23e6d5323a500b
I20260809 09:33:05.385905 720 ts_tablet_manager.cc:1939] T 9ea69a45821c40099c23e6d5323a500b P 2a843b4644cd4b2389cf749e5c3f2e58: Deleting consensus metadata
I20260809 09:33:05.386654 32316 catalog_manager.cc:4841] TS 2a843b4644cd4b2389cf749e5c3f2e58 (127.20.125.129:40235): tablet 9ea69a45821c40099c23e6d5323a500b (table test-table [id=f3a12d6c177347ea83fcaf9d2804aa05]) successfully deleted
Aug 09 09:33:05 dist-test-slave-7kzw krb5kdc[32272](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786267985, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 09:33:05 dist-test-slave-7kzw krb5kdc[32272](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.20.125.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 09:33:05 dist-test-slave-7kzw krb5kdc[32272](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.20.125.190@KRBTEST.COM, Server not found in Kerberos database
W20260809 09:33:05.415520 32409 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 09:33:05.408080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39934 (local address 127.20.125.190:43265)
0809 09:33:05.408239 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0809 09:33:05.408243 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 09:33:05.408504 (+ 261us) server_negotiation.cc:407] Connection header received
0809 09:33:05.408701 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:33:05.408705 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:33:05.408769 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:33:05.408871 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:33:05.409681 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:33:05.410286 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:33:05.411349 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:33:05.411606 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:33:05.411735 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:33:05.415376 (+ 3641us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39934: BlockingRecv error: recv got EOF from 127.0.0.1:39934 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Aug 09 09:33:05 dist-test-slave-7kzw krb5kdc[32272](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786267985, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.20.125.190@KRBTEST.COM
I20260809 09:33:05.427220 32409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:33:05.417366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39948 (local address 127.20.125.190:43265)
0809 09:33:05.417501 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0809 09:33:05.417505 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 09:33:05.417736 (+ 231us) server_negotiation.cc:407] Connection header received
0809 09:33:05.417905 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:33:05.417909 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:33:05.417956 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:33:05.418060 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:33:05.418772 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:33:05.419338 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:33:05.420193 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:33:05.420451 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:33:05.420574 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:33:05.423553 (+ 2979us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:33:05.423557 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:33:05.423560 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:33:05.423587 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:33:05.425507 (+ 1920us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:33:05.425938 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:33:05.425942 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:33:05.425943 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:33:05.426040 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:33:05.426314 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:33:05.426317 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:33:05.426319 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:33:05.426579 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:33:05.426725 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0809 09:33:05.426954 (+ 229us) server_negotiation.cc:299] Negotiation successful
0809 09:33:05.427067 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 09:33:05.430416 32334 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39948:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 09:33:05.430919 32334 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 09:33:05.436265 32545 tablet_service.cc:1467] Processing CreateTablet for tablet 4be71e03cccd48fd83236af7fb449bf7 (DEFAULT_TABLE table=test-table [id=f755e31ec05943cbbdad1999e5e21e46]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 09:33:05.436405 487 tablet_service.cc:1467] Processing CreateTablet for tablet 4be71e03cccd48fd83236af7fb449bf7 (DEFAULT_TABLE table=test-table [id=f755e31ec05943cbbdad1999e5e21e46]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 09:33:05.436612 32545 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4be71e03cccd48fd83236af7fb449bf7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 09:33:05.436666 487 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4be71e03cccd48fd83236af7fb449bf7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 09:33:05.436740 32762 tablet_service.cc:1467] Processing CreateTablet for tablet 4be71e03cccd48fd83236af7fb449bf7 (DEFAULT_TABLE table=test-table [id=f755e31ec05943cbbdad1999e5e21e46]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 09:33:05.437039 32762 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4be71e03cccd48fd83236af7fb449bf7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 09:33:05.439121 631 tablet_bootstrap.cc:492] T 4be71e03cccd48fd83236af7fb449bf7 P 2313d46fd7f84023b99b769f25a2bd37: Bootstrap starting.
I20260809 09:33:05.439180 630 tablet_bootstrap.cc:492] T 4be71e03cccd48fd83236af7fb449bf7 P b44b10e0ca7c4c9e89325813a36288d7: Bootstrap starting.
I20260809 09:33:05.439968 631 tablet_bootstrap.cc:654] T 4be71e03cccd48fd83236af7fb449bf7 P 2313d46fd7f84023b99b769f25a2bd37: Neither blocks nor log segments found. Creating new log.
I20260809 09:33:05.440039 630 tablet_bootstrap.cc:654] T 4be71e03cccd48fd83236af7fb449bf7 P b44b10e0ca7c4c9e89325813a36288d7: Neither blocks nor log segments found. Creating new log.
I20260809 09:33:05.440570 632 tablet_bootstrap.cc:492] T 4be71e03cccd48fd83236af7fb449bf7 P 2a843b4644cd4b2389cf749e5c3f2e58: Bootstrap starting.
I20260809 09:33:05.441181 631 tablet_bootstrap.cc:492] T 4be71e03cccd48fd83236af7fb449bf7 P 2313d46fd7f84023b99b769f25a2bd37: No bootstrap required, opened a new log
I20260809 09:33:05.441282 631 ts_tablet_manager.cc:1397] T 4be71e03cccd48fd83236af7fb449bf7 P 2313d46fd7f84023b99b769f25a2bd37: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20260809 09:33:05.441370 630 tablet_bootstrap.cc:492] T 4be71e03cccd48fd83236af7fb449bf7 P b44b10e0ca7c4c9e89325813a36288d7: No bootstrap required, opened a new log
I20260809 09:33:05.441457 630 ts_tablet_manager.cc:1397] T 4be71e03cccd48fd83236af7fb449bf7 P b44b10e0ca7c4c9e89325813a36288d7: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 09:33:05.441548 632 tablet_bootstrap.cc:654] T 4be71e03cccd48fd83236af7fb449bf7 P 2a843b4644cd4b2389cf749e5c3f2e58: Neither blocks nor log segments found. Creating new log.
I20260809 09:33:05.441779 631 raft_consensus.cc:348] T 4be71e03cccd48fd83236af7fb449bf7 P 2313d46fd7f84023b99b769f25a2bd37 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a843b4644cd4b2389cf749e5c3f2e58" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 40235 } } peers { permanent_uuid: "b44b10e0ca7c4c9e89325813a36288d7" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 33845 } } peers { permanent_uuid: "2313d46fd7f84023b99b769f25a2bd37" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 40493 } }
I20260809 09:33:05.441905 631 raft_consensus.cc:374] T 4be71e03cccd48fd83236af7fb449bf7 P 2313d46fd7f84023b99b769f25a2bd37 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 09:33:05.441944 631 raft_consensus.cc:729] T 4be71e03cccd48fd83236af7fb449bf7 P 2313d46fd7f84023b99b769f25a2bd37 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2313d46fd7f84023b99b769f25a2bd37, State: Initialized, Role: FOLLOWER
I20260809 09:33:05.441900 630 raft_consensus.cc:348] T 4be71e03cccd48fd83236af7fb449bf7 P b44b10e0ca7c4c9e89325813a36288d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a843b4644cd4b2389cf749e5c3f2e58" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 40235 } } peers { permanent_uuid: "b44b10e0ca7c4c9e89325813a36288d7" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 33845 } } peers { permanent_uuid: "2313d46fd7f84023b99b769f25a2bd37" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 40493 } }
I20260809 09:33:05.442015 630 raft_consensus.cc:374] T 4be71e03cccd48fd83236af7fb449bf7 P b44b10e0ca7c4c9e89325813a36288d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 09:33:05.442056 630 raft_consensus.cc:729] T 4be71e03cccd48fd83236af7fb449bf7 P b44b10e0ca7c4c9e89325813a36288d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b44b10e0ca7c4c9e89325813a36288d7, State: Initialized, Role: FOLLOWER
I20260809 09:33:05.442071 631 consensus_queue.cc:260] T 4be71e03cccd48fd83236af7fb449bf7 P 2313d46fd7f84023b99b769f25a2bd37 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a843b4644cd4b2389cf749e5c3f2e58" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 40235 } } peers { permanent_uuid: "b44b10e0ca7c4c9e89325813a36288d7" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 33845 } } peers { permanent_uuid: "2313d46fd7f84023b99b769f25a2bd37" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 40493 } }
I20260809 09:33:05.442193 630 consensus_queue.cc:260] T 4be71e03cccd48fd83236af7fb449bf7 P b44b10e0ca7c4c9e89325813a36288d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a843b4644cd4b2389cf749e5c3f2e58" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 40235 } } peers { permanent_uuid: "b44b10e0ca7c4c9e89325813a36288d7" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 33845 } } peers { permanent_uuid: "2313d46fd7f84023b99b769f25a2bd37" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 40493 } }
I20260809 09:33:05.442348 631 ts_tablet_manager.cc:1428] T 4be71e03cccd48fd83236af7fb449bf7 P 2313d46fd7f84023b99b769f25a2bd37: Time spent starting tablet: real 0.001s user 0.001s sys 0.001s
I20260809 09:33:05.442508 630 ts_tablet_manager.cc:1428] T 4be71e03cccd48fd83236af7fb449bf7 P b44b10e0ca7c4c9e89325813a36288d7: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260809 09:33:05.443068 632 tablet_bootstrap.cc:492] T 4be71e03cccd48fd83236af7fb449bf7 P 2a843b4644cd4b2389cf749e5c3f2e58: No bootstrap required, opened a new log
I20260809 09:33:05.443186 632 ts_tablet_manager.cc:1397] T 4be71e03cccd48fd83236af7fb449bf7 P 2a843b4644cd4b2389cf749e5c3f2e58: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260809 09:33:05.443604 632 raft_consensus.cc:348] T 4be71e03cccd48fd83236af7fb449bf7 P 2a843b4644cd4b2389cf749e5c3f2e58 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a843b4644cd4b2389cf749e5c3f2e58" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 40235 } } peers { permanent_uuid: "b44b10e0ca7c4c9e89325813a36288d7" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 33845 } } peers { permanent_uuid: "2313d46fd7f84023b99b769f25a2bd37" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 40493 } }
I20260809 09:33:05.443737 632 raft_consensus.cc:374] T 4be71e03cccd48fd83236af7fb449bf7 P 2a843b4644cd4b2389cf749e5c3f2e58 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 09:33:05.443778 632 raft_consensus.cc:729] T 4be71e03cccd48fd83236af7fb449bf7 P 2a843b4644cd4b2389cf749e5c3f2e58 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a843b4644cd4b2389cf749e5c3f2e58, State: Initialized, Role: FOLLOWER
I20260809 09:33:05.443915 632 consensus_queue.cc:260] T 4be71e03cccd48fd83236af7fb449bf7 P 2a843b4644cd4b2389cf749e5c3f2e58 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a843b4644cd4b2389cf749e5c3f2e58" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 40235 } } peers { permanent_uuid: "b44b10e0ca7c4c9e89325813a36288d7" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 33845 } } peers { permanent_uuid: "2313d46fd7f84023b99b769f25a2bd37" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 40493 } }
I20260809 09:33:05.444181 632 ts_tablet_manager.cc:1428] T 4be71e03cccd48fd83236af7fb449bf7 P 2a843b4644cd4b2389cf749e5c3f2e58: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260809 09:33:05.530035 696 raft_consensus.cc:482] T 4be71e03cccd48fd83236af7fb449bf7 P 2313d46fd7f84023b99b769f25a2bd37 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 09:33:05.530206 696 raft_consensus.cc:504] T 4be71e03cccd48fd83236af7fb449bf7 P 2313d46fd7f84023b99b769f25a2bd37 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a843b4644cd4b2389cf749e5c3f2e58" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 40235 } } peers { permanent_uuid: "b44b10e0ca7c4c9e89325813a36288d7" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 33845 } } peers { permanent_uuid: "2313d46fd7f84023b99b769f25a2bd37" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 40493 } }
I20260809 09:33:05.530686 696 leader_election.cc:288] T 4be71e03cccd48fd83236af7fb449bf7 P 2313d46fd7f84023b99b769f25a2bd37 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2a843b4644cd4b2389cf749e5c3f2e58 (127.20.125.129:40235), b44b10e0ca7c4c9e89325813a36288d7 (127.20.125.130:33845)
I20260809 09:33:05.531237 32577 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4be71e03cccd48fd83236af7fb449bf7" candidate_uuid: "2313d46fd7f84023b99b769f25a2bd37" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a843b4644cd4b2389cf749e5c3f2e58" is_pre_election: true
I20260809 09:33:05.531327 323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4be71e03cccd48fd83236af7fb449bf7" candidate_uuid: "2313d46fd7f84023b99b769f25a2bd37" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b44b10e0ca7c4c9e89325813a36288d7" is_pre_election: true
I20260809 09:33:05.531421 32577 raft_consensus.cc:2454] T 4be71e03cccd48fd83236af7fb449bf7 P 2a843b4644cd4b2389cf749e5c3f2e58 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2313d46fd7f84023b99b769f25a2bd37 in term 0.
I20260809 09:33:05.531527 323 raft_consensus.cc:2454] T 4be71e03cccd48fd83236af7fb449bf7 P b44b10e0ca7c4c9e89325813a36288d7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2313d46fd7f84023b99b769f25a2bd37 in term 0.
I20260809 09:33:05.531733 420 leader_election.cc:302] T 4be71e03cccd48fd83236af7fb449bf7 P 2313d46fd7f84023b99b769f25a2bd37 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2313d46fd7f84023b99b769f25a2bd37, 2a843b4644cd4b2389cf749e5c3f2e58; no voters:
I20260809 09:33:05.531927 696 raft_consensus.cc:2781] T 4be71e03cccd48fd83236af7fb449bf7 P 2313d46fd7f84023b99b769f25a2bd37 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 09:33:05.531997 696 raft_consensus.cc:482] T 4be71e03cccd48fd83236af7fb449bf7 P 2313d46fd7f84023b99b769f25a2bd37 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 09:33:05.532078 696 raft_consensus.cc:3037] T 4be71e03cccd48fd83236af7fb449bf7 P 2313d46fd7f84023b99b769f25a2bd37 [term 0 FOLLOWER]: Advancing to term 1
I20260809 09:33:05.533134 696 raft_consensus.cc:504] T 4be71e03cccd48fd83236af7fb449bf7 P 2313d46fd7f84023b99b769f25a2bd37 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a843b4644cd4b2389cf749e5c3f2e58" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 40235 } } peers { permanent_uuid: "b44b10e0ca7c4c9e89325813a36288d7" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 33845 } } peers { permanent_uuid: "2313d46fd7f84023b99b769f25a2bd37" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 40493 } }
I20260809 09:33:05.533457 696 leader_election.cc:288] T 4be71e03cccd48fd83236af7fb449bf7 P 2313d46fd7f84023b99b769f25a2bd37 [CANDIDATE]: Term 1 election: Requested vote from peers 2a843b4644cd4b2389cf749e5c3f2e58 (127.20.125.129:40235), b44b10e0ca7c4c9e89325813a36288d7 (127.20.125.130:33845)
I20260809 09:33:05.533847 32577 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4be71e03cccd48fd83236af7fb449bf7" candidate_uuid: "2313d46fd7f84023b99b769f25a2bd37" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a843b4644cd4b2389cf749e5c3f2e58"
I20260809 09:33:05.533958 32577 raft_consensus.cc:3037] T 4be71e03cccd48fd83236af7fb449bf7 P 2a843b4644cd4b2389cf749e5c3f2e58 [term 0 FOLLOWER]: Advancing to term 1
I20260809 09:33:05.533887 323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4be71e03cccd48fd83236af7fb449bf7" candidate_uuid: "2313d46fd7f84023b99b769f25a2bd37" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b44b10e0ca7c4c9e89325813a36288d7"
I20260809 09:33:05.534240 323 raft_consensus.cc:3037] T 4be71e03cccd48fd83236af7fb449bf7 P b44b10e0ca7c4c9e89325813a36288d7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 09:33:05.535149 32577 raft_consensus.cc:2454] T 4be71e03cccd48fd83236af7fb449bf7 P 2a843b4644cd4b2389cf749e5c3f2e58 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2313d46fd7f84023b99b769f25a2bd37 in term 1.
I20260809 09:33:05.535490 420 leader_election.cc:302] T 4be71e03cccd48fd83236af7fb449bf7 P 2313d46fd7f84023b99b769f25a2bd37 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2313d46fd7f84023b99b769f25a2bd37, 2a843b4644cd4b2389cf749e5c3f2e58; no voters:
I20260809 09:33:05.535658 696 raft_consensus.cc:2781] T 4be71e03cccd48fd83236af7fb449bf7 P 2313d46fd7f84023b99b769f25a2bd37 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 09:33:05.535713 323 raft_consensus.cc:2454] T 4be71e03cccd48fd83236af7fb449bf7 P b44b10e0ca7c4c9e89325813a36288d7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2313d46fd7f84023b99b769f25a2bd37 in term 1.
I20260809 09:33:05.535779 696 raft_consensus.cc:686] T 4be71e03cccd48fd83236af7fb449bf7 P 2313d46fd7f84023b99b769f25a2bd37 [term 1 LEADER]: Becoming Leader. State: Replica: 2313d46fd7f84023b99b769f25a2bd37, State: Running, Role: LEADER
I20260809 09:33:05.535930 696 consensus_queue.cc:237] T 4be71e03cccd48fd83236af7fb449bf7 P 2313d46fd7f84023b99b769f25a2bd37 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a843b4644cd4b2389cf749e5c3f2e58" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 40235 } } peers { permanent_uuid: "b44b10e0ca7c4c9e89325813a36288d7" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 33845 } } peers { permanent_uuid: "2313d46fd7f84023b99b769f25a2bd37" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 40493 } }
I20260809 09:33:05.537592 32334 catalog_manager.cc:5495] T 4be71e03cccd48fd83236af7fb449bf7 P 2313d46fd7f84023b99b769f25a2bd37 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2313d46fd7f84023b99b769f25a2bd37 (127.20.125.131). New cstate: current_term: 1 leader_uuid: "2313d46fd7f84023b99b769f25a2bd37" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a843b4644cd4b2389cf749e5c3f2e58" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 40235 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b44b10e0ca7c4c9e89325813a36288d7" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 33845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2313d46fd7f84023b99b769f25a2bd37" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 40493 } health_report { overall_health: HEALTHY } } }
I20260809 09:33:05.566762 625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:33:05.563310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38162 (local address 127.20.125.131:40493)
0809 09:33:05.563451 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0809 09:33:05.563455 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 09:33:05.563471 (+ 16us) server_negotiation.cc:407] Connection header received
0809 09:33:05.563615 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:33:05.563617 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:33:05.563685 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:33:05.563755 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 09:33:05.563981 (+ 226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:33:05.564491 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:33:05.565403 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:33:05.565651 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:33:05.565754 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:33:05.566098 (+ 344us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 09:33:05.566176 (+ 78us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 09:33:05.566239 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0809 09:33:05.566528 (+ 289us) server_negotiation.cc:299] Negotiation successful
0809 09:33:05.566596 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 09:33:05.569449 323 raft_consensus.cc:1261] T 4be71e03cccd48fd83236af7fb449bf7 P b44b10e0ca7c4c9e89325813a36288d7 [term 1 FOLLOWER]: Refusing update from remote peer 2313d46fd7f84023b99b769f25a2bd37: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 09:33:05.569490 32577 raft_consensus.cc:1261] T 4be71e03cccd48fd83236af7fb449bf7 P 2a843b4644cd4b2389cf749e5c3f2e58 [term 1 FOLLOWER]: Refusing update from remote peer 2313d46fd7f84023b99b769f25a2bd37: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 09:33:05.570010 719 consensus_queue.cc:1035] T 4be71e03cccd48fd83236af7fb449bf7 P 2313d46fd7f84023b99b769f25a2bd37 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b44b10e0ca7c4c9e89325813a36288d7" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 33845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 09:33:05.570197 696 consensus_queue.cc:1035] T 4be71e03cccd48fd83236af7fb449bf7 P 2313d46fd7f84023b99b769f25a2bd37 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2a843b4644cd4b2389cf749e5c3f2e58" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 40235 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 09:33:05.579986 32340 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39948:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 09:33:05.580160 32340 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39948:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 09:33:05.582237 32340 catalog_manager.cc:5782] T 00000000000000000000000000000000 P fdd08fcf074d4c108a4ea936f91b202d: Sending DeleteTablet for 3 replicas of tablet 4be71e03cccd48fd83236af7fb449bf7
I20260809 09:33:05.582877 487 tablet_service.cc:1514] Processing DeleteTablet for tablet 4be71e03cccd48fd83236af7fb449bf7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 09:33:05 UTC) from {username='oryx', principal='oryx/127.20.125.190@KRBTEST.COM'} at 127.0.0.1:38144
I20260809 09:33:05.583012 32762 tablet_service.cc:1514] Processing DeleteTablet for tablet 4be71e03cccd48fd83236af7fb449bf7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 09:33:05 UTC) from {username='oryx', principal='oryx/127.20.125.190@KRBTEST.COM'} at 127.0.0.1:56048
I20260809 09:33:05.583076 724 tablet_replica.cc:330] T 4be71e03cccd48fd83236af7fb449bf7 P 2313d46fd7f84023b99b769f25a2bd37: stopping tablet replica
I20260809 09:33:05.583207 724 raft_consensus.cc:2229] T 4be71e03cccd48fd83236af7fb449bf7 P 2313d46fd7f84023b99b769f25a2bd37 [term 1 LEADER]: Raft consensus shutting down.
I20260809 09:33:05.583503 724 raft_consensus.cc:2258] T 4be71e03cccd48fd83236af7fb449bf7 P 2313d46fd7f84023b99b769f25a2bd37 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 09:33:05.583488 32545 tablet_service.cc:1514] Processing DeleteTablet for tablet 4be71e03cccd48fd83236af7fb449bf7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 09:33:05 UTC) from {username='oryx', principal='oryx/127.20.125.190@KRBTEST.COM'} at 127.0.0.1:49832
I20260809 09:33:05.583673 720 tablet_replica.cc:330] T 4be71e03cccd48fd83236af7fb449bf7 P 2a843b4644cd4b2389cf749e5c3f2e58: stopping tablet replica
I20260809 09:33:05.583702 723 tablet_replica.cc:330] T 4be71e03cccd48fd83236af7fb449bf7 P b44b10e0ca7c4c9e89325813a36288d7: stopping tablet replica
I20260809 09:33:05.583775 720 raft_consensus.cc:2229] T 4be71e03cccd48fd83236af7fb449bf7 P 2a843b4644cd4b2389cf749e5c3f2e58 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 09:33:05.583822 723 raft_consensus.cc:2229] T 4be71e03cccd48fd83236af7fb449bf7 P b44b10e0ca7c4c9e89325813a36288d7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 09:33:05.583920 723 raft_consensus.cc:2258] T 4be71e03cccd48fd83236af7fb449bf7 P b44b10e0ca7c4c9e89325813a36288d7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 09:33:05.583910 720 raft_consensus.cc:2258] T 4be71e03cccd48fd83236af7fb449bf7 P 2a843b4644cd4b2389cf749e5c3f2e58 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 09:33:05.584612 720 ts_tablet_manager.cc:1905] T 4be71e03cccd48fd83236af7fb449bf7 P 2a843b4644cd4b2389cf749e5c3f2e58: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 09:33:05.584612 724 ts_tablet_manager.cc:1905] T 4be71e03cccd48fd83236af7fb449bf7 P 2313d46fd7f84023b99b769f25a2bd37: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 09:33:05.587164 720 ts_tablet_manager.cc:1918] T 4be71e03cccd48fd83236af7fb449bf7 P 2a843b4644cd4b2389cf749e5c3f2e58: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 09:33:05.587164 724 ts_tablet_manager.cc:1918] T 4be71e03cccd48fd83236af7fb449bf7 P 2313d46fd7f84023b99b769f25a2bd37: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 09:33:05.587240 724 log.cc:1196] T 4be71e03cccd48fd83236af7fb449bf7 P 2313d46fd7f84023b99b769f25a2bd37: Deleting WAL directory at /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786267941944493-20982-0/minicluster-data/ts-2/wal/wals/4be71e03cccd48fd83236af7fb449bf7
I20260809 09:33:05.587240 720 log.cc:1196] T 4be71e03cccd48fd83236af7fb449bf7 P 2a843b4644cd4b2389cf749e5c3f2e58: Deleting WAL directory at /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786267941944493-20982-0/minicluster-data/ts-0/wal/wals/4be71e03cccd48fd83236af7fb449bf7
I20260809 09:33:05.587525 720 ts_tablet_manager.cc:1939] T 4be71e03cccd48fd83236af7fb449bf7 P 2a843b4644cd4b2389cf749e5c3f2e58: Deleting consensus metadata
I20260809 09:33:05.587525 724 ts_tablet_manager.cc:1939] T 4be71e03cccd48fd83236af7fb449bf7 P 2313d46fd7f84023b99b769f25a2bd37: Deleting consensus metadata
I20260809 09:33:05.588191 32316 catalog_manager.cc:4841] TS 2a843b4644cd4b2389cf749e5c3f2e58 (127.20.125.129:40235): tablet 4be71e03cccd48fd83236af7fb449bf7 (table test-table [id=f755e31ec05943cbbdad1999e5e21e46]) successfully deleted
I20260809 09:33:05.588238 32318 catalog_manager.cc:4841] TS 2313d46fd7f84023b99b769f25a2bd37 (127.20.125.131:40493): tablet 4be71e03cccd48fd83236af7fb449bf7 (table test-table [id=f755e31ec05943cbbdad1999e5e21e46]) successfully deleted
I20260809 09:33:05.588882 723 ts_tablet_manager.cc:1905] T 4be71e03cccd48fd83236af7fb449bf7 P b44b10e0ca7c4c9e89325813a36288d7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 09:33:05.589471 32340 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39948:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 09:33:05.589915 32340 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 09:33:05.591531 723 ts_tablet_manager.cc:1918] T 4be71e03cccd48fd83236af7fb449bf7 P b44b10e0ca7c4c9e89325813a36288d7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 09:33:05.591605 723 log.cc:1196] T 4be71e03cccd48fd83236af7fb449bf7 P b44b10e0ca7c4c9e89325813a36288d7: Deleting WAL directory at /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786267941944493-20982-0/minicluster-data/ts-1/wal/wals/4be71e03cccd48fd83236af7fb449bf7
I20260809 09:33:05.591864 723 ts_tablet_manager.cc:1939] T 4be71e03cccd48fd83236af7fb449bf7 P b44b10e0ca7c4c9e89325813a36288d7: Deleting consensus metadata
I20260809 09:33:05.592607 32316 catalog_manager.cc:4841] TS b44b10e0ca7c4c9e89325813a36288d7 (127.20.125.130:33845): tablet 4be71e03cccd48fd83236af7fb449bf7 (table test-table [id=f755e31ec05943cbbdad1999e5e21e46]) successfully deleted
I20260809 09:33:05.596814 32545 tablet_service.cc:1467] Processing CreateTablet for tablet 93b0e19ea27446a78af1b5350da7c19a (DEFAULT_TABLE table=test-table [id=d001e89eb13d48388177045046c8401b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 09:33:05.596822 32762 tablet_service.cc:1467] Processing CreateTablet for tablet 93b0e19ea27446a78af1b5350da7c19a (DEFAULT_TABLE table=test-table [id=d001e89eb13d48388177045046c8401b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 09:33:05.597136 32545 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 93b0e19ea27446a78af1b5350da7c19a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 09:33:05.597142 32762 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 93b0e19ea27446a78af1b5350da7c19a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 09:33:05.599275 630 tablet_bootstrap.cc:492] T 93b0e19ea27446a78af1b5350da7c19a P b44b10e0ca7c4c9e89325813a36288d7: Bootstrap starting.
I20260809 09:33:05.599619 632 tablet_bootstrap.cc:492] T 93b0e19ea27446a78af1b5350da7c19a P 2a843b4644cd4b2389cf749e5c3f2e58: Bootstrap starting.
I20260809 09:33:05.600028 630 tablet_bootstrap.cc:654] T 93b0e19ea27446a78af1b5350da7c19a P b44b10e0ca7c4c9e89325813a36288d7: Neither blocks nor log segments found. Creating new log.
I20260809 09:33:05.600545 632 tablet_bootstrap.cc:654] T 93b0e19ea27446a78af1b5350da7c19a P 2a843b4644cd4b2389cf749e5c3f2e58: Neither blocks nor log segments found. Creating new log.
I20260809 09:33:05.601207 630 tablet_bootstrap.cc:492] T 93b0e19ea27446a78af1b5350da7c19a P b44b10e0ca7c4c9e89325813a36288d7: No bootstrap required, opened a new log
I20260809 09:33:05.601310 630 ts_tablet_manager.cc:1397] T 93b0e19ea27446a78af1b5350da7c19a P b44b10e0ca7c4c9e89325813a36288d7: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 09:33:05.601497 487 tablet_service.cc:1467] Processing CreateTablet for tablet 93b0e19ea27446a78af1b5350da7c19a (DEFAULT_TABLE table=test-table [id=d001e89eb13d48388177045046c8401b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 09:33:05.601790 487 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 93b0e19ea27446a78af1b5350da7c19a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 09:33:05.601737 630 raft_consensus.cc:348] T 93b0e19ea27446a78af1b5350da7c19a P b44b10e0ca7c4c9e89325813a36288d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a843b4644cd4b2389cf749e5c3f2e58" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 40235 } } peers { permanent_uuid: "b44b10e0ca7c4c9e89325813a36288d7" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 33845 } } peers { permanent_uuid: "2313d46fd7f84023b99b769f25a2bd37" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 40493 } }
I20260809 09:33:05.601866 630 raft_consensus.cc:374] T 93b0e19ea27446a78af1b5350da7c19a P b44b10e0ca7c4c9e89325813a36288d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 09:33:05.601907 630 raft_consensus.cc:729] T 93b0e19ea27446a78af1b5350da7c19a P b44b10e0ca7c4c9e89325813a36288d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b44b10e0ca7c4c9e89325813a36288d7, State: Initialized, Role: FOLLOWER
I20260809 09:33:05.602036 630 consensus_queue.cc:260] T 93b0e19ea27446a78af1b5350da7c19a P b44b10e0ca7c4c9e89325813a36288d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a843b4644cd4b2389cf749e5c3f2e58" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 40235 } } peers { permanent_uuid: "b44b10e0ca7c4c9e89325813a36288d7" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 33845 } } peers { permanent_uuid: "2313d46fd7f84023b99b769f25a2bd37" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 40493 } }
I20260809 09:33:05.602345 630 ts_tablet_manager.cc:1428] T 93b0e19ea27446a78af1b5350da7c19a P b44b10e0ca7c4c9e89325813a36288d7: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260809 09:33:05.604153 631 tablet_bootstrap.cc:492] T 93b0e19ea27446a78af1b5350da7c19a P 2313d46fd7f84023b99b769f25a2bd37: Bootstrap starting.
I20260809 09:33:05.604931 632 tablet_bootstrap.cc:492] T 93b0e19ea27446a78af1b5350da7c19a P 2a843b4644cd4b2389cf749e5c3f2e58: No bootstrap required, opened a new log
I20260809 09:33:05.605010 632 ts_tablet_manager.cc:1397] T 93b0e19ea27446a78af1b5350da7c19a P 2a843b4644cd4b2389cf749e5c3f2e58: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.001s
I20260809 09:33:05.605101 631 tablet_bootstrap.cc:654] T 93b0e19ea27446a78af1b5350da7c19a P 2313d46fd7f84023b99b769f25a2bd37: Neither blocks nor log segments found. Creating new log.
I20260809 09:33:05.605440 632 raft_consensus.cc:348] T 93b0e19ea27446a78af1b5350da7c19a P 2a843b4644cd4b2389cf749e5c3f2e58 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a843b4644cd4b2389cf749e5c3f2e58" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 40235 } } peers { permanent_uuid: "b44b10e0ca7c4c9e89325813a36288d7" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 33845 } } peers { permanent_uuid: "2313d46fd7f84023b99b769f25a2bd37" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 40493 } }
I20260809 09:33:05.605583 632 raft_consensus.cc:374] T 93b0e19ea27446a78af1b5350da7c19a P 2a843b4644cd4b2389cf749e5c3f2e58 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 09:33:05.605618 632 raft_consensus.cc:729] T 93b0e19ea27446a78af1b5350da7c19a P 2a843b4644cd4b2389cf749e5c3f2e58 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a843b4644cd4b2389cf749e5c3f2e58, State: Initialized, Role: FOLLOWER
I20260809 09:33:05.605731 632 consensus_queue.cc:260] T 93b0e19ea27446a78af1b5350da7c19a P 2a843b4644cd4b2389cf749e5c3f2e58 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a843b4644cd4b2389cf749e5c3f2e58" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 40235 } } peers { permanent_uuid: "b44b10e0ca7c4c9e89325813a36288d7" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 33845 } } peers { permanent_uuid: "2313d46fd7f84023b99b769f25a2bd37" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 40493 } }
I20260809 09:33:05.605973 632 ts_tablet_manager.cc:1428] T 93b0e19ea27446a78af1b5350da7c19a P 2a843b4644cd4b2389cf749e5c3f2e58: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260809 09:33:05.606220 631 tablet_bootstrap.cc:492] T 93b0e19ea27446a78af1b5350da7c19a P 2313d46fd7f84023b99b769f25a2bd37: No bootstrap required, opened a new log
I20260809 09:33:05.606302 631 ts_tablet_manager.cc:1397] T 93b0e19ea27446a78af1b5350da7c19a P 2313d46fd7f84023b99b769f25a2bd37: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20260809 09:33:05.606773 631 raft_consensus.cc:348] T 93b0e19ea27446a78af1b5350da7c19a P 2313d46fd7f84023b99b769f25a2bd37 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a843b4644cd4b2389cf749e5c3f2e58" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 40235 } } peers { permanent_uuid: "b44b10e0ca7c4c9e89325813a36288d7" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 33845 } } peers { permanent_uuid: "2313d46fd7f84023b99b769f25a2bd37" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 40493 } }
I20260809 09:33:05.606894 631 raft_consensus.cc:374] T 93b0e19ea27446a78af1b5350da7c19a P 2313d46fd7f84023b99b769f25a2bd37 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 09:33:05.606937 631 raft_consensus.cc:729] T 93b0e19ea27446a78af1b5350da7c19a P 2313d46fd7f84023b99b769f25a2bd37 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2313d46fd7f84023b99b769f25a2bd37, State: Initialized, Role: FOLLOWER
I20260809 09:33:05.607084 631 consensus_queue.cc:260] T 93b0e19ea27446a78af1b5350da7c19a P 2313d46fd7f84023b99b769f25a2bd37 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a843b4644cd4b2389cf749e5c3f2e58" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 40235 } } peers { permanent_uuid: "b44b10e0ca7c4c9e89325813a36288d7" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 33845 } } peers { permanent_uuid: "2313d46fd7f84023b99b769f25a2bd37" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 40493 } }
I20260809 09:33:05.607391 631 ts_tablet_manager.cc:1428] T 93b0e19ea27446a78af1b5350da7c19a P 2313d46fd7f84023b99b769f25a2bd37: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260809 09:33:05.805236 644 raft_consensus.cc:482] T 93b0e19ea27446a78af1b5350da7c19a P b44b10e0ca7c4c9e89325813a36288d7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 09:33:05.805413 644 raft_consensus.cc:504] T 93b0e19ea27446a78af1b5350da7c19a P b44b10e0ca7c4c9e89325813a36288d7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a843b4644cd4b2389cf749e5c3f2e58" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 40235 } } peers { permanent_uuid: "b44b10e0ca7c4c9e89325813a36288d7" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 33845 } } peers { permanent_uuid: "2313d46fd7f84023b99b769f25a2bd37" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 40493 } }
I20260809 09:33:05.805846 644 leader_election.cc:288] T 93b0e19ea27446a78af1b5350da7c19a P b44b10e0ca7c4c9e89325813a36288d7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2a843b4644cd4b2389cf749e5c3f2e58 (127.20.125.129:40235), 2313d46fd7f84023b99b769f25a2bd37 (127.20.125.131:40493)
I20260809 09:33:05.806353 32577 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "93b0e19ea27446a78af1b5350da7c19a" candidate_uuid: "b44b10e0ca7c4c9e89325813a36288d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a843b4644cd4b2389cf749e5c3f2e58" is_pre_election: true
I20260809 09:33:05.806439 519 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "93b0e19ea27446a78af1b5350da7c19a" candidate_uuid: "b44b10e0ca7c4c9e89325813a36288d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2313d46fd7f84023b99b769f25a2bd37" is_pre_election: true
I20260809 09:33:05.806548 32577 raft_consensus.cc:2454] T 93b0e19ea27446a78af1b5350da7c19a P 2a843b4644cd4b2389cf749e5c3f2e58 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b44b10e0ca7c4c9e89325813a36288d7 in term 0.
I20260809 09:33:05.806622 519 raft_consensus.cc:2454] T 93b0e19ea27446a78af1b5350da7c19a P 2313d46fd7f84023b99b769f25a2bd37 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b44b10e0ca7c4c9e89325813a36288d7 in term 0.
I20260809 09:33:05.807016 32692 leader_election.cc:302] T 93b0e19ea27446a78af1b5350da7c19a P b44b10e0ca7c4c9e89325813a36288d7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 2313d46fd7f84023b99b769f25a2bd37, 2a843b4644cd4b2389cf749e5c3f2e58, b44b10e0ca7c4c9e89325813a36288d7; no voters:
I20260809 09:33:05.807257 644 raft_consensus.cc:2781] T 93b0e19ea27446a78af1b5350da7c19a P b44b10e0ca7c4c9e89325813a36288d7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 09:33:05.807322 644 raft_consensus.cc:482] T 93b0e19ea27446a78af1b5350da7c19a P b44b10e0ca7c4c9e89325813a36288d7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 09:33:05.807353 644 raft_consensus.cc:3037] T 93b0e19ea27446a78af1b5350da7c19a P b44b10e0ca7c4c9e89325813a36288d7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 09:33:05.808354 644 raft_consensus.cc:504] T 93b0e19ea27446a78af1b5350da7c19a P b44b10e0ca7c4c9e89325813a36288d7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a843b4644cd4b2389cf749e5c3f2e58" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 40235 } } peers { permanent_uuid: "b44b10e0ca7c4c9e89325813a36288d7" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 33845 } } peers { permanent_uuid: "2313d46fd7f84023b99b769f25a2bd37" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 40493 } }
I20260809 09:33:05.808703 644 leader_election.cc:288] T 93b0e19ea27446a78af1b5350da7c19a P b44b10e0ca7c4c9e89325813a36288d7 [CANDIDATE]: Term 1 election: Requested vote from peers 2a843b4644cd4b2389cf749e5c3f2e58 (127.20.125.129:40235), 2313d46fd7f84023b99b769f25a2bd37 (127.20.125.131:40493)
I20260809 09:33:05.809006 32577 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "93b0e19ea27446a78af1b5350da7c19a" candidate_uuid: "b44b10e0ca7c4c9e89325813a36288d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a843b4644cd4b2389cf749e5c3f2e58"
I20260809 09:33:05.809134 32577 raft_consensus.cc:3037] T 93b0e19ea27446a78af1b5350da7c19a P 2a843b4644cd4b2389cf749e5c3f2e58 [term 0 FOLLOWER]: Advancing to term 1
I20260809 09:33:05.809111 519 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "93b0e19ea27446a78af1b5350da7c19a" candidate_uuid: "b44b10e0ca7c4c9e89325813a36288d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2313d46fd7f84023b99b769f25a2bd37"
I20260809 09:33:05.809293 519 raft_consensus.cc:3037] T 93b0e19ea27446a78af1b5350da7c19a P 2313d46fd7f84023b99b769f25a2bd37 [term 0 FOLLOWER]: Advancing to term 1
I20260809 09:33:05.810103 32577 raft_consensus.cc:2454] T 93b0e19ea27446a78af1b5350da7c19a P 2a843b4644cd4b2389cf749e5c3f2e58 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b44b10e0ca7c4c9e89325813a36288d7 in term 1.
I20260809 09:33:05.810341 519 raft_consensus.cc:2454] T 93b0e19ea27446a78af1b5350da7c19a P 2313d46fd7f84023b99b769f25a2bd37 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b44b10e0ca7c4c9e89325813a36288d7 in term 1.
I20260809 09:33:05.810403 32685 leader_election.cc:302] T 93b0e19ea27446a78af1b5350da7c19a P b44b10e0ca7c4c9e89325813a36288d7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a843b4644cd4b2389cf749e5c3f2e58, b44b10e0ca7c4c9e89325813a36288d7; no voters:
I20260809 09:33:05.810645 644 raft_consensus.cc:2781] T 93b0e19ea27446a78af1b5350da7c19a P b44b10e0ca7c4c9e89325813a36288d7 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 09:33:05.810761 644 raft_consensus.cc:686] T 93b0e19ea27446a78af1b5350da7c19a P b44b10e0ca7c4c9e89325813a36288d7 [term 1 LEADER]: Becoming Leader. State: Replica: b44b10e0ca7c4c9e89325813a36288d7, State: Running, Role: LEADER
I20260809 09:33:05.810916 644 consensus_queue.cc:237] T 93b0e19ea27446a78af1b5350da7c19a P b44b10e0ca7c4c9e89325813a36288d7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a843b4644cd4b2389cf749e5c3f2e58" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 40235 } } peers { permanent_uuid: "b44b10e0ca7c4c9e89325813a36288d7" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 33845 } } peers { permanent_uuid: "2313d46fd7f84023b99b769f25a2bd37" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 40493 } }
I20260809 09:33:05.812644 32334 catalog_manager.cc:5495] T 93b0e19ea27446a78af1b5350da7c19a P b44b10e0ca7c4c9e89325813a36288d7 reported cstate change: term changed from 0 to 1, leader changed from <none> to b44b10e0ca7c4c9e89325813a36288d7 (127.20.125.130). New cstate: current_term: 1 leader_uuid: "b44b10e0ca7c4c9e89325813a36288d7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a843b4644cd4b2389cf749e5c3f2e58" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 40235 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b44b10e0ca7c4c9e89325813a36288d7" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 33845 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2313d46fd7f84023b99b769f25a2bd37" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 40493 } health_report { overall_health: UNKNOWN } } }
I20260809 09:33:05.850927 628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:33:05.847684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56072 (local address 127.20.125.130:33845)
0809 09:33:05.847898 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0809 09:33:05.847902 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 09:33:05.847915 (+ 13us) server_negotiation.cc:407] Connection header received
0809 09:33:05.847998 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:33:05.848001 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:33:05.848050 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:33:05.848111 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 09:33:05.848418 (+ 307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:33:05.848909 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:33:05.849715 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:33:05.849944 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:33:05.850035 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:33:05.850350 (+ 315us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 09:33:05.850425 (+ 75us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 09:33:05.850597 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0809 09:33:05.850689 (+ 92us) server_negotiation.cc:299] Negotiation successful
0809 09:33:05.850747 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260809 09:33:05.857599 519 raft_consensus.cc:1261] T 93b0e19ea27446a78af1b5350da7c19a P 2313d46fd7f84023b99b769f25a2bd37 [term 1 FOLLOWER]: Refusing update from remote peer b44b10e0ca7c4c9e89325813a36288d7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 09:33:05.857980 32577 raft_consensus.cc:1261] T 93b0e19ea27446a78af1b5350da7c19a P 2a843b4644cd4b2389cf749e5c3f2e58 [term 1 FOLLOWER]: Refusing update from remote peer b44b10e0ca7c4c9e89325813a36288d7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 09:33:05.858042 644 consensus_queue.cc:1035] T 93b0e19ea27446a78af1b5350da7c19a P b44b10e0ca7c4c9e89325813a36288d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2313d46fd7f84023b99b769f25a2bd37" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 40493 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 09:33:05.858330 642 consensus_queue.cc:1035] T 93b0e19ea27446a78af1b5350da7c19a P b44b10e0ca7c4c9e89325813a36288d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2a843b4644cd4b2389cf749e5c3f2e58" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 40235 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 09:33:05.891422 32334 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39948:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 09:33:05.891606 32334 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39948:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 09:33:05.893702 32334 catalog_manager.cc:5782] T 00000000000000000000000000000000 P fdd08fcf074d4c108a4ea936f91b202d: Sending DeleteTablet for 3 replicas of tablet 93b0e19ea27446a78af1b5350da7c19a
I20260809 09:33:05.894392 32545 tablet_service.cc:1514] Processing DeleteTablet for tablet 93b0e19ea27446a78af1b5350da7c19a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 09:33:05 UTC) from {username='oryx', principal='oryx/127.20.125.190@KRBTEST.COM'} at 127.0.0.1:49832
I20260809 09:33:05.894400 487 tablet_service.cc:1514] Processing DeleteTablet for tablet 93b0e19ea27446a78af1b5350da7c19a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 09:33:05 UTC) from {username='oryx', principal='oryx/127.20.125.190@KRBTEST.COM'} at 127.0.0.1:38144
I20260809 09:33:05.894447 32762 tablet_service.cc:1514] Processing DeleteTablet for tablet 93b0e19ea27446a78af1b5350da7c19a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 09:33:05 UTC) from {username='oryx', principal='oryx/127.20.125.190@KRBTEST.COM'} at 127.0.0.1:56048
I20260809 09:33:05.894640 723 tablet_replica.cc:330] T 93b0e19ea27446a78af1b5350da7c19a P b44b10e0ca7c4c9e89325813a36288d7: stopping tablet replica
I20260809 09:33:05.894662 720 tablet_replica.cc:330] T 93b0e19ea27446a78af1b5350da7c19a P 2a843b4644cd4b2389cf749e5c3f2e58: stopping tablet replica
I20260809 09:33:05.894773 723 raft_consensus.cc:2229] T 93b0e19ea27446a78af1b5350da7c19a P b44b10e0ca7c4c9e89325813a36288d7 [term 1 LEADER]: Raft consensus shutting down.
I20260809 09:33:05.894810 720 raft_consensus.cc:2229] T 93b0e19ea27446a78af1b5350da7c19a P 2a843b4644cd4b2389cf749e5c3f2e58 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 09:33:05.894954 720 raft_consensus.cc:2258] T 93b0e19ea27446a78af1b5350da7c19a P 2a843b4644cd4b2389cf749e5c3f2e58 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 09:33:05.894989 723 raft_consensus.cc:2258] T 93b0e19ea27446a78af1b5350da7c19a P b44b10e0ca7c4c9e89325813a36288d7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 09:33:05.895153 724 tablet_replica.cc:330] T 93b0e19ea27446a78af1b5350da7c19a P 2313d46fd7f84023b99b769f25a2bd37: stopping tablet replica
I20260809 09:33:05.895292 724 raft_consensus.cc:2229] T 93b0e19ea27446a78af1b5350da7c19a P 2313d46fd7f84023b99b769f25a2bd37 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 09:33:05.895453 724 raft_consensus.cc:2258] T 93b0e19ea27446a78af1b5350da7c19a P 2313d46fd7f84023b99b769f25a2bd37 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 09:33:05.895720 720 ts_tablet_manager.cc:1905] T 93b0e19ea27446a78af1b5350da7c19a P 2a843b4644cd4b2389cf749e5c3f2e58: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 09:33:05.896211 723 ts_tablet_manager.cc:1905] T 93b0e19ea27446a78af1b5350da7c19a P b44b10e0ca7c4c9e89325813a36288d7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 09:33:05.896238 724 ts_tablet_manager.cc:1905] T 93b0e19ea27446a78af1b5350da7c19a P 2313d46fd7f84023b99b769f25a2bd37: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 09:33:05.898187 724 ts_tablet_manager.cc:1918] T 93b0e19ea27446a78af1b5350da7c19a P 2313d46fd7f84023b99b769f25a2bd37: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 09:33:05.898243 724 log.cc:1196] T 93b0e19ea27446a78af1b5350da7c19a P 2313d46fd7f84023b99b769f25a2bd37: Deleting WAL directory at /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786267941944493-20982-0/minicluster-data/ts-2/wal/wals/93b0e19ea27446a78af1b5350da7c19a
I20260809 09:33:05.898437 724 ts_tablet_manager.cc:1939] T 93b0e19ea27446a78af1b5350da7c19a P 2313d46fd7f84023b99b769f25a2bd37: Deleting consensus metadata
I20260809 09:33:05.898535 720 ts_tablet_manager.cc:1918] T 93b0e19ea27446a78af1b5350da7c19a P 2a843b4644cd4b2389cf749e5c3f2e58: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 09:33:05.898627 720 log.cc:1196] T 93b0e19ea27446a78af1b5350da7c19a P 2a843b4644cd4b2389cf749e5c3f2e58: Deleting WAL directory at /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786267941944493-20982-0/minicluster-data/ts-0/wal/wals/93b0e19ea27446a78af1b5350da7c19a
I20260809 09:33:05.898892 723 ts_tablet_manager.cc:1918] T 93b0e19ea27446a78af1b5350da7c19a P b44b10e0ca7c4c9e89325813a36288d7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 09:33:05.898916 720 ts_tablet_manager.cc:1939] T 93b0e19ea27446a78af1b5350da7c19a P 2a843b4644cd4b2389cf749e5c3f2e58: Deleting consensus metadata
I20260809 09:33:05.898967 723 log.cc:1196] T 93b0e19ea27446a78af1b5350da7c19a P b44b10e0ca7c4c9e89325813a36288d7: Deleting WAL directory at /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786267941944493-20982-0/minicluster-data/ts-1/wal/wals/93b0e19ea27446a78af1b5350da7c19a
I20260809 09:33:05.899164 32318 catalog_manager.cc:4841] TS 2313d46fd7f84023b99b769f25a2bd37 (127.20.125.131:40493): tablet 93b0e19ea27446a78af1b5350da7c19a (table test-table [id=d001e89eb13d48388177045046c8401b]) successfully deleted
I20260809 09:33:05.899233 723 ts_tablet_manager.cc:1939] T 93b0e19ea27446a78af1b5350da7c19a P b44b10e0ca7c4c9e89325813a36288d7: Deleting consensus metadata
I20260809 09:33:05.899762 20982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu with pid 32446
I20260809 09:33:05.905740 32316 catalog_manager.cc:4841] TS 2a843b4644cd4b2389cf749e5c3f2e58 (127.20.125.129:40235): tablet 93b0e19ea27446a78af1b5350da7c19a (table test-table [id=d001e89eb13d48388177045046c8401b]) successfully deleted
I20260809 09:33:05.911280 32316 catalog_manager.cc:4841] TS b44b10e0ca7c4c9e89325813a36288d7 (127.20.125.130:33845): tablet 93b0e19ea27446a78af1b5350da7c19a (table test-table [id=d001e89eb13d48388177045046c8401b]) successfully deleted
I20260809 09:33:05.914232 20982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu with pid 32658
I20260809 09:33:05.929114 20982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu with pid 399
I20260809 09:33:05.951558 20982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu with pid 32291
2026-08-09T09:33:05Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (3650 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786267941944493-20982-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 09:33:06 dist-test-slave-7kzw krb5kdc[763](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 09:33:06 dist-test-slave-7kzw krb5kdc[763](info): set up 2 sockets
Aug 09 09:33:06 dist-test-slave-7kzw krb5kdc[763](info): commencing operation
krb5kdc: starting...
W20260809 09:33:07.186339 20982 mini_kdc.cc:121] Time spent starting KDC: real 1.205s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786267941944493-20982-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786267941944493-20982-0/krb5kdc/test-user.keytab.
Aug 09 09:33:07 dist-test-slave-7kzw krb5kdc[763](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786267987, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T09:33:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T09:33:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.125.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.125.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.125.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.190.keytab.
Entry for principal kudu/127.20.125.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.190.keytab.
I20260809 09:33:07.327836 20982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
/tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786267941944493-20982-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786267941944493-20982-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786267941944493-20982-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786267941944493-20982-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.125.190:34067
--webserver_interface=127.20.125.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.190.keytab
--principal=kudu/127.20.125.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.125.148:35329
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.125.190:34067
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786267941944493-20982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786267941944493-20982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786267941944493-20982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 09:33:07.443277 781 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 09:33:07.443579 781 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 09:33:07.443624 781 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 09:33:07.447822 781 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 09:33:07.447889 781 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 09:33:07.447909 781 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 09:33:07.447926 781 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 09:33:07.447942 781 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 09:33:07.453395 781 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.125.148:35329
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786267941944493-20982-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786267941944493-20982-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.125.190:34067
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.190.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.125.190:34067
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786267941944493-20982-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.125.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.781
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786267941944493-20982-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 08:38:55 UTC on ccycloud.quasar-njjleu.root.comops.site
build id 4575935
I20260809 09:33:07.454587 781 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 09:33:07.455657 781 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 09:33:07.464318 790 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:33:07.464330 781 server_base.cc:1034] running on GCE node
W20260809 09:33:07.464332 787 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:33:07.464355 792 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:33:07.465227 781 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 09:33:07.466270 781 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 09:33:07.467466 781 hybrid_clock.cc:648] HybridClock initialized: now 1786267987467468 us; error 51 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-09T09:33:07Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1528 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786267941944493-20982-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 09:33:07 dist-test-slave-7kzw krb5kdc[799](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 09:33:07 dist-test-slave-7kzw krb5kdc[799](info): set up 2 sockets
Aug 09 09:33:07 dist-test-slave-7kzw krb5kdc[799](info): commencing operation
krb5kdc: starting...
W20260809 09:33:08.753726 20982 mini_kdc.cc:121] Time spent starting KDC: real 1.244s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786267941944493-20982-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786267941944493-20982-0/krb5kdc/test-user.keytab.
Aug 09 09:33:08 dist-test-slave-7kzw krb5kdc[799](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786267988, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T09:33:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T09:33:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.125.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.125.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.125.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.190.keytab.
Entry for principal kudu/127.20.125.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.190.keytab.
I20260809 09:33:08.890681 20982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
/tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786267941944493-20982-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786267941944493-20982-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786267941944493-20982-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786267941944493-20982-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.125.190:40061
--webserver_interface=127.20.125.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.190.keytab
--principal=kudu/127.20.125.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.125.148:32893
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.125.190:40061
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786267941944493-20982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786267941944493-20982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786267941944493-20982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 09:33:09.005914 822 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 09:33:09.006234 822 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 09:33:09.006320 822 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 09:33:09.010401 822 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 09:33:09.010522 822 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 09:33:09.010618 822 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 09:33:09.010654 822 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 09:33:09.010689 822 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 09:33:09.016072 822 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.125.148:32893
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786267941944493-20982-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786267941944493-20982-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.125.190:40061
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.190.keytab
--principal=kudu/127.20.125.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.125.190:40061
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786267941944493-20982-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.125.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.822
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786267941944493-20982-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 08:38:55 UTC on ccycloud.quasar-njjleu.root.comops.site
build id 4575935
I20260809 09:33:09.017191 822 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 09:33:09.018247 822 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 09:33:09.026916 822 server_base.cc:1034] running on GCE node
W20260809 09:33:09.026883 829 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:33:09.026899 834 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:33:09.026907 831 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:33:09.027842 822 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 09:33:09.028928 822 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 09:33:09.030138 822 hybrid_clock.cc:648] HybridClock initialized: now 1786267989030118 us; error 33 us; skew 500 ppm
Aug 09 09:33:09 dist-test-slave-7kzw krb5kdc[799](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786267989, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 09:33:09.033716 822 init.cc:377] Logged in from keytab as kudu/127.20.125.190@KRBTEST.COM (short username kudu)
I20260809 09:33:09.034775 822 webserver.cc:460] Webserver started at http://127.20.125.190:44961/ using document root <none> and password file <none>
I20260809 09:33:09.035406 822 fs_manager.cc:359] Metadata directory not provided
I20260809 09:33:09.035480 822 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 09:33:09.035730 822 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 09:33:09.038059 822 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786267941944493-20982-0/minicluster-data/master-0/data/instance:
uuid: "8142e866b2d44844b1801487350325db"
format_stamp: "Formatted at 2026-08-09 09:33:09 on dist-test-slave-7kzw"
server_key: "999a608933579d5cacd23ba4a0557aac"
server_key_iv: "d6e5ebd81d6abd7ca7d29e9ba25a9d9d"
server_key_version: "encryptionkey@0"
I20260809 09:33:09.038726 822 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786267941944493-20982-0/minicluster-data/master-0/wal/instance:
uuid: "8142e866b2d44844b1801487350325db"
format_stamp: "Formatted at 2026-08-09 09:33:09 on dist-test-slave-7kzw"
server_key: "999a608933579d5cacd23ba4a0557aac"
server_key_iv: "d6e5ebd81d6abd7ca7d29e9ba25a9d9d"
server_key_version: "encryptionkey@0"
I20260809 09:33:09.042912 822 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260809 09:33:09.045516 847 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 09:33:09.046931 822 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 09:33:09.047076 822 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786267941944493-20982-0/minicluster-data/master-0/data,/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786267941944493-20982-0/minicluster-data/master-0/wal
uuid: "8142e866b2d44844b1801487350325db"
format_stamp: "Formatted at 2026-08-09 09:33:09 on dist-test-slave-7kzw"
server_key: "999a608933579d5cacd23ba4a0557aac"
server_key_iv: "d6e5ebd81d6abd7ca7d29e9ba25a9d9d"
server_key_version: "encryptionkey@0"
I20260809 09:33:09.047192 822 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786267941944493-20982-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786267941944493-20982-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786267941944493-20982-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 09:33:09.064785 822 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 09:33:09.068094 822 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 09:33:09.068315 822 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 09:33:09.076620 822 rpc_server.cc:307] RPC server started. Bound to: 127.20.125.190:40061
I20260809 09:33:09.076645 914 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.125.190:40061 every 8 connection(s)
I20260809 09:33:09.077672 822 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786267941944493-20982-0/minicluster-data/master-0/data/info.pb
I20260809 09:33:09.078742 20982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu as pid 822
I20260809 09:33:09.078907 20982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786267941944493-20982-0/minicluster-data/master-0/wal/instance
I20260809 09:33:09.079282 20982 external_mini_cluster.cc:1428] Setting key b3b04aa3197db77686f8118e8a7f5086
I20260809 09:33:09.081535 915 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 09:33:09.088907 915 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8142e866b2d44844b1801487350325db: Bootstrap starting.
Aug 09 09:33:09 dist-test-slave-7kzw krb5kdc[799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786267988, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.125.190@KRBTEST.COM
I20260809 09:33:09.091365 915 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8142e866b2d44844b1801487350325db: Neither blocks nor log segments found. Creating new log.
I20260809 09:33:09.092180 915 log.cc:824] T 00000000000000000000000000000000 P 8142e866b2d44844b1801487350325db: Log is configured to *not* fsync() on all Append() calls
I20260809 09:33:09.094308 915 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8142e866b2d44844b1801487350325db: No bootstrap required, opened a new log
I20260809 09:33:09.095036 919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:33:09.080910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38898 (local address 127.20.125.190:40061)
0809 09:33:09.081374 (+ 464us) server_negotiation.cc:206] Beginning negotiation
0809 09:33:09.081394 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0809 09:33:09.081444 (+ 50us) server_negotiation.cc:407] Connection header received
0809 09:33:09.082197 (+ 753us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:33:09.082215 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:33:09.082632 (+ 417us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:33:09.082930 (+ 298us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:33:09.083964 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:33:09.084982 (+ 1018us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:33:09.086273 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:33:09.086809 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:33:09.086932 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:33:09.089981 (+ 3049us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:33:09.089987 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:33:09.089998 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:33:09.090025 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:33:09.091914 (+ 1889us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:33:09.092386 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:33:09.092391 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:33:09.092396 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:33:09.092508 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:33:09.092806 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:33:09.092810 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:33:09.092812 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:33:09.093240 (+ 428us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:33:09.093435 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0809 09:33:09.093762 (+ 327us) server_negotiation.cc:299] Negotiation successful
0809 09:33:09.094038 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":314,"thread_start_us":164,"threads_started":1}
I20260809 09:33:09.097702 915 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8142e866b2d44844b1801487350325db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8142e866b2d44844b1801487350325db" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 40061 } }
I20260809 09:33:09.097956 915 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8142e866b2d44844b1801487350325db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 09:33:09.098048 915 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8142e866b2d44844b1801487350325db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8142e866b2d44844b1801487350325db, State: Initialized, Role: FOLLOWER
I20260809 09:33:09.098542 915 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8142e866b2d44844b1801487350325db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8142e866b2d44844b1801487350325db" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 40061 } }
I20260809 09:33:09.098729 915 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8142e866b2d44844b1801487350325db [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 09:33:09.098826 915 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8142e866b2d44844b1801487350325db [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 09:33:09.098959 915 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8142e866b2d44844b1801487350325db [term 0 FOLLOWER]: Advancing to term 1
I20260809 09:33:09.099917 915 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8142e866b2d44844b1801487350325db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8142e866b2d44844b1801487350325db" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 40061 } }
I20260809 09:33:09.100275 915 leader_election.cc:302] T 00000000000000000000000000000000 P 8142e866b2d44844b1801487350325db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8142e866b2d44844b1801487350325db; no voters:
I20260809 09:33:09.100653 915 leader_election.cc:288] T 00000000000000000000000000000000 P 8142e866b2d44844b1801487350325db [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 09:33:09.100792 922 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8142e866b2d44844b1801487350325db [term 1 FOLLOWER]: Leader election won for term 1
I20260809 09:33:09.101022 922 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8142e866b2d44844b1801487350325db [term 1 LEADER]: Becoming Leader. State: Replica: 8142e866b2d44844b1801487350325db, State: Running, Role: LEADER
I20260809 09:33:09.101321 922 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8142e866b2d44844b1801487350325db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8142e866b2d44844b1801487350325db" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 40061 } }
I20260809 09:33:09.102032 915 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8142e866b2d44844b1801487350325db [sys.catalog]: configured and running, proceeding with master startup.
I20260809 09:33:09.103235 924 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8142e866b2d44844b1801487350325db [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8142e866b2d44844b1801487350325db. Latest consensus state: current_term: 1 leader_uuid: "8142e866b2d44844b1801487350325db" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8142e866b2d44844b1801487350325db" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 40061 } } }
I20260809 09:33:09.103389 924 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8142e866b2d44844b1801487350325db [sys.catalog]: This master's current role is: LEADER
I20260809 09:33:09.103698 923 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8142e866b2d44844b1801487350325db [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8142e866b2d44844b1801487350325db" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8142e866b2d44844b1801487350325db" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 40061 } } }
I20260809 09:33:09.103777 936 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 09:33:09.103803 923 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8142e866b2d44844b1801487350325db [sys.catalog]: This master's current role is: LEADER
I20260809 09:33:09.106843 936 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 09:33:09.111958 936 catalog_manager.cc:1347] Generated new cluster ID: 38eff1d8bb6c4a8a98f1a2df53329d85
I20260809 09:33:09.112030 936 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 09:33:09.133766 936 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 09:33:09.134879 936 catalog_manager.cc:1504] Loading token signing keys...
I20260809 09:33:09.148785 936 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8142e866b2d44844b1801487350325db: Generated new TSK 0
I20260809 09:33:09.149505 936 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.20.125.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.125.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.125.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.129.keytab.
Entry for principal kudu/127.20.125.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.129.keytab.
I20260809 09:33:09.182649 20982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
/tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786267941944493-20982-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786267941944493-20982-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786267941944493-20982-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786267941944493-20982-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.125.129:0
--local_ip_for_outbound_sockets=127.20.125.129
--webserver_interface=127.20.125.129
--webserver_port=0
--tserver_master_addrs=127.20.125.190:40061
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.129.keytab
--principal=kudu/127.20.125.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.125.148:32893
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786267941944493-20982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786267941944493-20982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786267941944493-20982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 09:33:09.310022 952 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 09:33:09.310516 952 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 09:33:09.310588 952 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 09:33:09.317960 952 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 09:33:09.318071 952 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 09:33:09.318171 952 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.125.129
I20260809 09:33:09.328374 952 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.125.148:32893
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786267941944493-20982-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786267941944493-20982-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.129.keytab
--principal=kudu/127.20.125.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.125.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786267941944493-20982-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.125.129
--webserver_port=0
--tserver_master_addrs=127.20.125.190:40061
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.952
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.125.129
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786267941944493-20982-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 08:38:55 UTC on ccycloud.quasar-njjleu.root.comops.site
build id 4575935
I20260809 09:33:09.329924 952 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 09:33:09.331449 952 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 09:33:09.343597 960 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:33:09.343606 959 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:33:09.343660 962 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:33:09.343945 952 server_base.cc:1034] running on GCE node
I20260809 09:33:09.344373 952 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 09:33:09.345005 952 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 09:33:09.346192 952 hybrid_clock.cc:648] HybridClock initialized: now 1786267989346155 us; error 47 us; skew 500 ppm
Aug 09 09:33:09 dist-test-slave-7kzw krb5kdc[799](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786267989, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 09:33:09.349095 952 init.cc:377] Logged in from keytab as kudu/127.20.125.129@KRBTEST.COM (short username kudu)
I20260809 09:33:09.350026 952 webserver.cc:460] Webserver started at http://127.20.125.129:42395/ using document root <none> and password file <none>
I20260809 09:33:09.350718 952 fs_manager.cc:359] Metadata directory not provided
I20260809 09:33:09.350785 952 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 09:33:09.350952 952 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 09:33:09.353085 952 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786267941944493-20982-0/minicluster-data/ts-0/data/instance:
uuid: "f9f2d5ca5c78445089aab1e5e8da693d"
format_stamp: "Formatted at 2026-08-09 09:33:09 on dist-test-slave-7kzw"
server_key: "18920d02f253dc3b5fa3616cf23cb4ab"
server_key_iv: "e093d14288406e8547ec1a298e4c17c7"
server_key_version: "encryptionkey@0"
I20260809 09:33:09.353602 952 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786267941944493-20982-0/minicluster-data/ts-0/wal/instance:
uuid: "f9f2d5ca5c78445089aab1e5e8da693d"
format_stamp: "Formatted at 2026-08-09 09:33:09 on dist-test-slave-7kzw"
server_key: "18920d02f253dc3b5fa3616cf23cb4ab"
server_key_iv: "e093d14288406e8547ec1a298e4c17c7"
server_key_version: "encryptionkey@0"
I20260809 09:33:09.357364 952 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260809 09:33:09.359803 972 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 09:33:09.360999 952 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 09:33:09.361096 952 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786267941944493-20982-0/minicluster-data/ts-0/data,/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786267941944493-20982-0/minicluster-data/ts-0/wal
uuid: "f9f2d5ca5c78445089aab1e5e8da693d"
format_stamp: "Formatted at 2026-08-09 09:33:09 on dist-test-slave-7kzw"
server_key: "18920d02f253dc3b5fa3616cf23cb4ab"
server_key_iv: "e093d14288406e8547ec1a298e4c17c7"
server_key_version: "encryptionkey@0"
I20260809 09:33:09.361200 952 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786267941944493-20982-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786267941944493-20982-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786267941944493-20982-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 09:33:09.375787 952 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 09:33:09.379228 952 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 09:33:09.379383 952 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 09:33:09.379980 952 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 09:33:09.380728 952 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 09:33:09.380774 952 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 09:33:09.380813 952 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 09:33:09.380828 952 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 09:33:09.389420 952 rpc_server.cc:307] RPC server started. Bound to: 127.20.125.129:36363
I20260809 09:33:09.389503 1129 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.125.129:36363 every 8 connection(s)
I20260809 09:33:09.390343 952 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786267941944493-20982-0/minicluster-data/ts-0/data/info.pb
I20260809 09:33:09.394827 20982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu as pid 952
I20260809 09:33:09.394968 20982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786267941944493-20982-0/minicluster-data/ts-0/wal/instance
I20260809 09:33:09.395354 20982 external_mini_cluster.cc:1428] Setting key 32b82728d879f61175894b46d8169e81
Aug 09 09:33:09 dist-test-slave-7kzw krb5kdc[799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786267989, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.129@KRBTEST.COM for kudu/127.20.125.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.20.125.130@KRBTEST.COM; defaulting to no policy
I20260809 09:33:09.407366 919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:33:09.392261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.129:37527 (local address 127.20.125.190:40061)
0809 09:33:09.392411 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0809 09:33:09.392415 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 09:33:09.393291 (+ 876us) server_negotiation.cc:407] Connection header received
0809 09:33:09.394828 (+ 1537us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:33:09.394831 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:33:09.394891 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:33:09.395002 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:33:09.396756 (+ 1754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:33:09.397403 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:33:09.399368 (+ 1965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:33:09.399823 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:33:09.399957 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:33:09.403272 (+ 3315us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:33:09.403276 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:33:09.403278 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:33:09.403303 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:33:09.405009 (+ 1706us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:33:09.405606 (+ 597us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:33:09.405610 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:33:09.405612 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:33:09.405712 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:33:09.406101 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:33:09.406104 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:33:09.406106 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:33:09.406304 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:33:09.406422 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0809 09:33:09.407002 (+ 580us) server_negotiation.cc:299] Negotiation successful
0809 09:33:09.407143 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260809 09:33:09.408190 1134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:33:09.392627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:40061 (local address 127.20.125.129:37527)
0809 09:33:09.393127 (+ 500us) negotiation.cc:107] Waiting for socket to connect
0809 09:33:09.393170 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 09:33:09.394616 (+ 1446us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:33:09.395333 (+ 717us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:33:09.395343 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:33:09.395891 (+ 548us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:33:09.396511 (+ 620us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:33:09.396530 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:33:09.397571 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:33:09.397575 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:33:09.399209 (+ 1634us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:33:09.399226 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:33:09.399995 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:33:09.399998 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:33:09.400161 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:33:09.400829 (+ 668us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 09:33:09.400847 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:33:09.403074 (+ 2227us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 09:33:09.405166 (+ 2092us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:33:09.405178 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:33:09.405194 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:33:09.405485 (+ 291us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:33:09.405819 (+ 334us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:33:09.405822 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:33:09.405825 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:33:09.405986 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:33:09.406421 (+ 435us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 09:33:09.406428 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 09:33:09.406736 (+ 308us) client_negotiation.cc:769] Sending connection context
0809 09:33:09.406938 (+ 202us) client_negotiation.cc:240] Negotiation successful
0809 09:33:09.407307 (+ 369us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":301,"thread_start_us":124,"threads_started":1}
I20260809 09:33:09.409572 1131 heartbeater.cc:347] Connected to a master server at 127.20.125.190:40061
I20260809 09:33:09.409847 1131 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.125.130@KRBTEST.COM" created.
I20260809 09:33:09.410517 1131 heartbeater.cc:511] Master 127.20.125.190:40061 requested a full tablet report, sending...
I20260809 09:33:09.412398 868 ts_manager.cc:194] Registered new tserver with Master: f9f2d5ca5c78445089aab1e5e8da693d (127.20.125.129:36363)
I20260809 09:33:09.413652 868 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.125.129@KRBTEST.COM'} at 127.20.125.129:37527
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.125.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.130.keytab.
Entry for principal kudu/127.20.125.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.130.keytab.
I20260809 09:33:09.429483 20982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
/tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786267941944493-20982-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786267941944493-20982-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786267941944493-20982-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786267941944493-20982-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.125.130:0
--local_ip_for_outbound_sockets=127.20.125.130
--webserver_interface=127.20.125.130
--webserver_port=0
--tserver_master_addrs=127.20.125.190:40061
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.130.keytab
--principal=kudu/127.20.125.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.125.148:32893
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786267941944493-20982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786267941944493-20982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786267941944493-20982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 09:33:09.543563 1139 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 09:33:09.543886 1139 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 09:33:09.543931 1139 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 09:33:09.547976 1139 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 09:33:09.548039 1139 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 09:33:09.548098 1139 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.125.130
I20260809 09:33:09.553629 1139 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.125.148:32893
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786267941944493-20982-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786267941944493-20982-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.130.keytab
--principal=kudu/127.20.125.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.125.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786267941944493-20982-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.125.130
--webserver_port=0
--tserver_master_addrs=127.20.125.190:40061
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1139
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.125.130
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786267941944493-20982-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 08:38:55 UTC on ccycloud.quasar-njjleu.root.comops.site
build id 4575935
I20260809 09:33:09.554766 1139 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 09:33:09.555773 1139 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 09:33:09.565758 1152 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:33:09.565812 1146 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:33:09.565788 1149 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:33:09.566789 1139 server_base.cc:1034] running on GCE node
I20260809 09:33:09.567255 1139 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 09:33:09.568022 1139 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 09:33:09.569255 1139 hybrid_clock.cc:648] HybridClock initialized: now 1786267989569227 us; error 54 us; skew 500 ppm
Aug 09 09:33:09 dist-test-slave-7kzw krb5kdc[799](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786267989, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 09:33:09.572500 1139 init.cc:377] Logged in from keytab as kudu/127.20.125.130@KRBTEST.COM (short username kudu)
I20260809 09:33:09.573410 1139 webserver.cc:460] Webserver started at http://127.20.125.130:46235/ using document root <none> and password file <none>
I20260809 09:33:09.573997 1139 fs_manager.cc:359] Metadata directory not provided
I20260809 09:33:09.574065 1139 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 09:33:09.574292 1139 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 09:33:09.576397 1139 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786267941944493-20982-0/minicluster-data/ts-1/data/instance:
uuid: "97e14f4b033f480da0eace143d724235"
format_stamp: "Formatted at 2026-08-09 09:33:09 on dist-test-slave-7kzw"
server_key: "431f48e90f1c41a3be95971265a520ff"
server_key_iv: "45363598bfa32bd8500f0fd8ae7315aa"
server_key_version: "encryptionkey@0"
I20260809 09:33:09.576979 1139 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786267941944493-20982-0/minicluster-data/ts-1/wal/instance:
uuid: "97e14f4b033f480da0eace143d724235"
format_stamp: "Formatted at 2026-08-09 09:33:09 on dist-test-slave-7kzw"
server_key: "431f48e90f1c41a3be95971265a520ff"
server_key_iv: "45363598bfa32bd8500f0fd8ae7315aa"
server_key_version: "encryptionkey@0"
I20260809 09:33:09.580860 1139 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.002s
I20260809 09:33:09.583400 1163 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 09:33:09.584642 1139 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 09:33:09.584784 1139 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786267941944493-20982-0/minicluster-data/ts-1/data,/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786267941944493-20982-0/minicluster-data/ts-1/wal
uuid: "97e14f4b033f480da0eace143d724235"
format_stamp: "Formatted at 2026-08-09 09:33:09 on dist-test-slave-7kzw"
server_key: "431f48e90f1c41a3be95971265a520ff"
server_key_iv: "45363598bfa32bd8500f0fd8ae7315aa"
server_key_version: "encryptionkey@0"
I20260809 09:33:09.584898 1139 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786267941944493-20982-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786267941944493-20982-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786267941944493-20982-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 09:33:09.596659 1139 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 09:33:09.599920 1139 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 09:33:09.600123 1139 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 09:33:09.600718 1139 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 09:33:09.601569 1139 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 09:33:09.601641 1139 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 09:33:09.601716 1139 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 09:33:09.601769 1139 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 09:33:09.611565 1139 rpc_server.cc:307] RPC server started. Bound to: 127.20.125.130:35317
I20260809 09:33:09.611591 1348 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.125.130:35317 every 8 connection(s)
I20260809 09:33:09.612519 1139 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786267941944493-20982-0/minicluster-data/ts-1/data/info.pb
I20260809 09:33:09.616869 20982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu as pid 1139
I20260809 09:33:09.616997 20982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786267941944493-20982-0/minicluster-data/ts-1/wal/instance
I20260809 09:33:09.617312 20982 external_mini_cluster.cc:1428] Setting key 693562c325366b8994bfbd384f8f0ad5
Aug 09 09:33:09 dist-test-slave-7kzw krb5kdc[799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786267989, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.130@KRBTEST.COM for kudu/127.20.125.190@KRBTEST.COM
I20260809 09:33:09.628520 919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:33:09.614346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.130:44923 (local address 127.20.125.190:40061)
0809 09:33:09.614518 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0809 09:33:09.614523 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 09:33:09.615421 (+ 898us) server_negotiation.cc:407] Connection header received
0809 09:33:09.616847 (+ 1426us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:33:09.616850 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:33:09.616911 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:33:09.616997 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:33:09.618626 (+ 1629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:33:09.619244 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:33:09.620989 (+ 1745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:33:09.621367 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:33:09.621491 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:33:09.624487 (+ 2996us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:33:09.624490 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:33:09.624492 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:33:09.624515 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:33:09.626072 (+ 1557us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:33:09.626794 (+ 722us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:33:09.626797 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:33:09.626799 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:33:09.626892 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:33:09.627344 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:33:09.627346 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:33:09.627347 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:33:09.627530 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:33:09.627602 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0809 09:33:09.628284 (+ 682us) server_negotiation.cc:299] Negotiation successful
0809 09:33:09.628390 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
WARNING: no policy specified for kudu/127.20.125.131@KRBTEST.COM; defaulting to no policy
I20260809 09:33:09.629860 1353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:33:09.614803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:40061 (local address 127.20.125.130:44923)
0809 09:33:09.615265 (+ 462us) negotiation.cc:107] Waiting for socket to connect
0809 09:33:09.615311 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 09:33:09.616596 (+ 1285us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:33:09.617300 (+ 704us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:33:09.617308 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:33:09.617824 (+ 516us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:33:09.618437 (+ 613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:33:09.618458 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:33:09.619389 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:33:09.619395 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:33:09.620846 (+ 1451us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:33:09.620862 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:33:09.621537 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:33:09.621539 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:33:09.621684 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:33:09.622326 (+ 642us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 09:33:09.622343 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:33:09.624298 (+ 1955us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 09:33:09.626200 (+ 1902us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:33:09.626209 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:33:09.626228 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:33:09.626684 (+ 456us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:33:09.626985 (+ 301us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:33:09.626989 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:33:09.626991 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:33:09.627220 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:33:09.627643 (+ 423us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 09:33:09.627650 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 09:33:09.628002 (+ 352us) client_negotiation.cc:769] Sending connection context
0809 09:33:09.628239 (+ 237us) client_negotiation.cc:240] Negotiation successful
0809 09:33:09.628757 (+ 518us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":274,"thread_start_us":104,"threads_started":1}
I20260809 09:33:09.631142 1349 heartbeater.cc:347] Connected to a master server at 127.20.125.190:40061
I20260809 09:33:09.631402 1349 heartbeater.cc:464] Registering TS with master...
I20260809 09:33:09.631942 1349 heartbeater.cc:511] Master 127.20.125.190:40061 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.125.131@KRBTEST.COM" created.
I20260809 09:33:09.633282 868 ts_manager.cc:194] Registered new tserver with Master: 97e14f4b033f480da0eace143d724235 (127.20.125.130:35317)
I20260809 09:33:09.634042 868 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.125.130@KRBTEST.COM'} at 127.20.125.130:44923
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.125.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.131.keytab.
Entry for principal kudu/127.20.125.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.131.keytab.
I20260809 09:33:09.650236 20982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
/tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786267941944493-20982-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786267941944493-20982-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786267941944493-20982-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786267941944493-20982-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.125.131:0
--local_ip_for_outbound_sockets=127.20.125.131
--webserver_interface=127.20.125.131
--webserver_port=0
--tserver_master_addrs=127.20.125.190:40061
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.131.keytab
--principal=kudu/127.20.125.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.125.148:32893
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786267941944493-20982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786267941944493-20982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786267941944493-20982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 09:33:09.767609 1357 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 09:33:09.767926 1357 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 09:33:09.768122 1357 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 09:33:09.773861 1357 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 09:33:09.773980 1357 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 09:33:09.774080 1357 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.125.131
I20260809 09:33:09.780405 1357 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.125.148:32893
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786267941944493-20982-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786267941944493-20982-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.131.keytab
--principal=kudu/127.20.125.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.125.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786267941944493-20982-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.125.131
--webserver_port=0
--tserver_master_addrs=127.20.125.190:40061
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1357
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.125.131
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786267941944493-20982-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 08:38:55 UTC on ccycloud.quasar-njjleu.root.comops.site
build id 4575935
I20260809 09:33:09.781545 1357 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 09:33:09.782699 1357 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 09:33:09.793151 1366 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:33:09.793116 1370 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:33:09.793129 1357 server_base.cc:1034] running on GCE node
W20260809 09:33:09.793133 1365 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:33:09.793818 1357 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 09:33:09.794514 1357 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 09:33:09.795744 1357 hybrid_clock.cc:648] HybridClock initialized: now 1786267989795731 us; error 27 us; skew 500 ppm
Aug 09 09:33:09 dist-test-slave-7kzw krb5kdc[799](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786267989, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 09:33:09.798672 1357 init.cc:377] Logged in from keytab as kudu/127.20.125.131@KRBTEST.COM (short username kudu)
I20260809 09:33:09.799749 1357 webserver.cc:460] Webserver started at http://127.20.125.131:42597/ using document root <none> and password file <none>
I20260809 09:33:09.800369 1357 fs_manager.cc:359] Metadata directory not provided
I20260809 09:33:09.800475 1357 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 09:33:09.800734 1357 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 09:33:09.803249 1357 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786267941944493-20982-0/minicluster-data/ts-2/data/instance:
uuid: "7539421cfc5f47f3ab4e3290c87c9a23"
format_stamp: "Formatted at 2026-08-09 09:33:09 on dist-test-slave-7kzw"
server_key: "39b47792096b2e07cce3a7fa980d9e93"
server_key_iv: "bba5d4bf88fd871ba7fa1b1ab86e2bbb"
server_key_version: "encryptionkey@0"
I20260809 09:33:09.803923 1357 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786267941944493-20982-0/minicluster-data/ts-2/wal/instance:
uuid: "7539421cfc5f47f3ab4e3290c87c9a23"
format_stamp: "Formatted at 2026-08-09 09:33:09 on dist-test-slave-7kzw"
server_key: "39b47792096b2e07cce3a7fa980d9e93"
server_key_iv: "bba5d4bf88fd871ba7fa1b1ab86e2bbb"
server_key_version: "encryptionkey@0"
I20260809 09:33:09.808744 1357 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 09:33:09.811504 1388 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 09:33:09.812906 1357 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 09:33:09.813110 1357 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786267941944493-20982-0/minicluster-data/ts-2/data,/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786267941944493-20982-0/minicluster-data/ts-2/wal
uuid: "7539421cfc5f47f3ab4e3290c87c9a23"
format_stamp: "Formatted at 2026-08-09 09:33:09 on dist-test-slave-7kzw"
server_key: "39b47792096b2e07cce3a7fa980d9e93"
server_key_iv: "bba5d4bf88fd871ba7fa1b1ab86e2bbb"
server_key_version: "encryptionkey@0"
I20260809 09:33:09.813252 1357 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786267941944493-20982-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786267941944493-20982-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786267941944493-20982-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 09:33:09.868587 1357 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 09:33:09.872110 1357 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 09:33:09.872349 1357 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 09:33:09.873027 1357 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 09:33:09.874240 1357 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 09:33:09.874320 1357 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 09:33:09.874389 1357 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 09:33:09.874419 1357 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 09:33:09.883534 1357 rpc_server.cc:307] RPC server started. Bound to: 127.20.125.131:43633
I20260809 09:33:09.883556 1554 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.125.131:43633 every 8 connection(s)
I20260809 09:33:09.884686 1357 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786267941944493-20982-0/minicluster-data/ts-2/data/info.pb
I20260809 09:33:09.888278 20982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu as pid 1357
I20260809 09:33:09.888422 20982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786267941944493-20982-0/minicluster-data/ts-2/wal/instance
I20260809 09:33:09.888778 20982 external_mini_cluster.cc:1428] Setting key 139e5db82341042de6c98dd0b227b4b9
Aug 09 09:33:09 dist-test-slave-7kzw krb5kdc[799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786267989, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.131@KRBTEST.COM for kudu/127.20.125.190@KRBTEST.COM
I20260809 09:33:09.899132 919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:33:09.886797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.131:43777 (local address 127.20.125.190:40061)
0809 09:33:09.886988 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0809 09:33:09.886992 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 09:33:09.887834 (+ 842us) server_negotiation.cc:407] Connection header received
0809 09:33:09.889555 (+ 1721us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:33:09.889558 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:33:09.889614 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:33:09.889724 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:33:09.891128 (+ 1404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:33:09.891687 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:33:09.892817 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:33:09.893039 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:33:09.893148 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:33:09.895706 (+ 2558us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:33:09.895710 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:33:09.895713 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:33:09.895742 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:33:09.897056 (+ 1314us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:33:09.897573 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:33:09.897577 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:33:09.897578 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:33:09.897653 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:33:09.897998 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:33:09.898002 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:33:09.898003 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:33:09.898179 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:33:09.898291 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0809 09:33:09.898837 (+ 546us) server_negotiation.cc:299] Negotiation successful
0809 09:33:09.898959 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260809 09:33:09.899883 1559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:33:09.887157 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:40061 (local address 127.20.125.131:43777)
0809 09:33:09.887673 (+ 516us) negotiation.cc:107] Waiting for socket to connect
0809 09:33:09.887719 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 09:33:09.889347 (+ 1628us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:33:09.889909 (+ 562us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:33:09.889916 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:33:09.890263 (+ 347us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:33:09.890978 (+ 715us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:33:09.891006 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:33:09.891813 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:33:09.891816 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:33:09.892676 (+ 860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:33:09.892694 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:33:09.893186 (+ 492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:33:09.893190 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:33:09.893366 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:33:09.893973 (+ 607us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 09:33:09.893989 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:33:09.895547 (+ 1558us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 09:33:09.897174 (+ 1627us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:33:09.897179 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:33:09.897194 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:33:09.897456 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:33:09.897751 (+ 295us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:33:09.897754 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:33:09.897756 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:33:09.897893 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:33:09.898291 (+ 398us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 09:33:09.898299 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 09:33:09.898592 (+ 293us) client_negotiation.cc:769] Sending connection context
0809 09:33:09.898779 (+ 187us) client_negotiation.cc:240] Negotiation successful
0809 09:33:09.899087 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":309,"thread_start_us":142,"threads_started":1}
I20260809 09:33:09.901096 1555 heartbeater.cc:347] Connected to a master server at 127.20.125.190:40061
I20260809 09:33:09.901408 1555 heartbeater.cc:464] Registering TS with master...
I20260809 09:33:09.901969 1555 heartbeater.cc:511] Master 127.20.125.190:40061 requested a full tablet report, sending...
I20260809 09:33:09.903230 868 ts_manager.cc:194] Registered new tserver with Master: 7539421cfc5f47f3ab4e3290c87c9a23 (127.20.125.131:43633)
I20260809 09:33:09.903821 868 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.125.131@KRBTEST.COM'} at 127.20.125.131:43777
I20260809 09:33:09.915577 20982 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.20.125.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.20.125.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.20.125.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786267941944493-20982-0/krb5kdc/oryx_127.20.125.129.keytab.
Entry for principal oryx/127.20.125.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786267941944493-20982-0/krb5kdc/oryx_127.20.125.129.keytab.
WARNING: no policy specified for oryx/127.20.125.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.20.125.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.20.125.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786267941944493-20982-0/krb5kdc/oryx_127.20.125.130.keytab.
Entry for principal oryx/127.20.125.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786267941944493-20982-0/krb5kdc/oryx_127.20.125.130.keytab.
WARNING: no policy specified for oryx/127.20.125.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.20.125.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.20.125.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786267941944493-20982-0/krb5kdc/oryx_127.20.125.131.keytab.
Entry for principal oryx/127.20.125.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786267941944493-20982-0/krb5kdc/oryx_127.20.125.131.keytab.
I20260809 09:33:10.001739 20982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu with pid 952
I20260809 09:33:10.015415 20982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu with pid 1139
I20260809 09:33:10.037326 20982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu with pid 1357
I20260809 09:33:10.055970 20982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu with pid 822
I20260809 09:33:10.066286 20982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
/tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786267941944493-20982-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786267941944493-20982-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786267941944493-20982-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786267941944493-20982-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.125.190:40061
--webserver_interface=127.20.125.190
--webserver_port=44961
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.190.keytab
--principal=kudu/127.20.125.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.125.148:32893
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.125.190:40061
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786267941944493-20982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786267941944493-20982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786267941944493-20982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 09:33:10.180145 1574 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 09:33:10.180505 1574 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 09:33:10.180583 1574 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 09:33:10.184654 1574 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 09:33:10.184767 1574 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 09:33:10.184806 1574 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 09:33:10.184845 1574 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 09:33:10.184880 1574 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 09:33:10.190299 1574 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.125.148:32893
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786267941944493-20982-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786267941944493-20982-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.125.190:40061
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.190.keytab
--principal=kudu/127.20.125.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.125.190:40061
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786267941944493-20982-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.125.190
--webserver_port=44961
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1574
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786267941944493-20982-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 08:38:55 UTC on ccycloud.quasar-njjleu.root.comops.site
build id 4575935
I20260809 09:33:10.191471 1574 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 09:33:10.192942 1574 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 09:33:10.202312 1580 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:33:10.202280 1583 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:33:10.202299 1581 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:33:10.202656 1574 server_base.cc:1034] running on GCE node
I20260809 09:33:10.203516 1574 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 09:33:10.204761 1574 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 09:33:10.205976 1574 hybrid_clock.cc:648] HybridClock initialized: now 1786267990205911 us; error 82 us; skew 500 ppm
Aug 09 09:33:10 dist-test-slave-7kzw krb5kdc[799](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786267990, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 09:33:10.209365 1574 init.cc:377] Logged in from keytab as kudu/127.20.125.190@KRBTEST.COM (short username kudu)
I20260809 09:33:10.210381 1574 webserver.cc:460] Webserver started at http://127.20.125.190:44961/ using document root <none> and password file <none>
I20260809 09:33:10.211030 1574 fs_manager.cc:359] Metadata directory not provided
I20260809 09:33:10.211164 1574 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 09:33:10.215278 1574 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260809 09:33:10.217670 1595 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 09:33:10.219204 1574 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 09:33:10.219380 1574 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786267941944493-20982-0/minicluster-data/master-0/data,/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786267941944493-20982-0/minicluster-data/master-0/wal
uuid: "8142e866b2d44844b1801487350325db"
format_stamp: "Formatted at 2026-08-09 09:33:09 on dist-test-slave-7kzw"
server_key: "999a608933579d5cacd23ba4a0557aac"
server_key_iv: "d6e5ebd81d6abd7ca7d29e9ba25a9d9d"
server_key_version: "encryptionkey@0"
I20260809 09:33:10.220010 1574 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786267941944493-20982-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786267941944493-20982-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786267941944493-20982-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 09:33:10.244956 1574 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 09:33:10.248710 1574 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 09:33:10.248944 1574 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 09:33:10.256453 1574 rpc_server.cc:307] RPC server started. Bound to: 127.20.125.190:40061
I20260809 09:33:10.256481 1668 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.125.190:40061 every 8 connection(s)
I20260809 09:33:10.257532 1574 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786267941944493-20982-0/minicluster-data/master-0/data/info.pb
I20260809 09:33:10.262274 1670 sys_catalog.cc:263] Verifying existing consensus state
I20260809 09:33:10.263892 20982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu as pid 1574
I20260809 09:33:10.264209 1670 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8142e866b2d44844b1801487350325db: Bootstrap starting.
I20260809 09:33:10.264442 20982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
/tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786267941944493-20982-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786267941944493-20982-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786267941944493-20982-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786267941944493-20982-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.125.129:36363
--local_ip_for_outbound_sockets=127.20.125.129
--tserver_master_addrs=127.20.125.190:40061
--webserver_port=42395
--webserver_interface=127.20.125.129
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.129.keytab
--principal=kudu/127.20.125.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.125.148:32893
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.20.125.129
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786267941944493-20982-0/krb5kdc/oryx_127.20.125.129.keytab with env {KRB5CCNAME=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786267941944493-20982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786267941944493-20982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786267941944493-20982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 09:33:10.282565 1670 log.cc:824] T 00000000000000000000000000000000 P 8142e866b2d44844b1801487350325db: Log is configured to *not* fsync() on all Append() calls
I20260809 09:33:10.289667 1670 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8142e866b2d44844b1801487350325db: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 09:33:10.290081 1670 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8142e866b2d44844b1801487350325db: Bootstrap complete.
I20260809 09:33:10.294411 1670 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8142e866b2d44844b1801487350325db [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8142e866b2d44844b1801487350325db" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 40061 } }
I20260809 09:33:10.294906 1670 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8142e866b2d44844b1801487350325db [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8142e866b2d44844b1801487350325db, State: Initialized, Role: FOLLOWER
I20260809 09:33:10.295519 1670 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8142e866b2d44844b1801487350325db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8142e866b2d44844b1801487350325db" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 40061 } }
I20260809 09:33:10.295742 1670 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8142e866b2d44844b1801487350325db [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 09:33:10.295816 1670 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8142e866b2d44844b1801487350325db [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 09:33:10.295912 1670 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8142e866b2d44844b1801487350325db [term 1 FOLLOWER]: Advancing to term 2
I20260809 09:33:10.298679 1670 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8142e866b2d44844b1801487350325db [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8142e866b2d44844b1801487350325db" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 40061 } }
I20260809 09:33:10.299026 1670 leader_election.cc:302] T 00000000000000000000000000000000 P 8142e866b2d44844b1801487350325db [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8142e866b2d44844b1801487350325db; no voters:
I20260809 09:33:10.299418 1670 leader_election.cc:288] T 00000000000000000000000000000000 P 8142e866b2d44844b1801487350325db [CANDIDATE]: Term 2 election: Requested vote from peers
I20260809 09:33:10.299515 1675 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8142e866b2d44844b1801487350325db [term 2 FOLLOWER]: Leader election won for term 2
I20260809 09:33:10.299719 1675 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8142e866b2d44844b1801487350325db [term 2 LEADER]: Becoming Leader. State: Replica: 8142e866b2d44844b1801487350325db, State: Running, Role: LEADER
I20260809 09:33:10.300112 1675 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8142e866b2d44844b1801487350325db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8142e866b2d44844b1801487350325db" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 40061 } }
I20260809 09:33:10.300697 1670 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8142e866b2d44844b1801487350325db [sys.catalog]: configured and running, proceeding with master startup.
I20260809 09:33:10.301083 1675 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8142e866b2d44844b1801487350325db [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8142e866b2d44844b1801487350325db. Latest consensus state: current_term: 2 leader_uuid: "8142e866b2d44844b1801487350325db" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8142e866b2d44844b1801487350325db" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 40061 } } }
I20260809 09:33:10.301208 1675 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8142e866b2d44844b1801487350325db [sys.catalog]: This master's current role is: LEADER
I20260809 09:33:10.301592 1681 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 09:33:10.301678 1676 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8142e866b2d44844b1801487350325db [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "8142e866b2d44844b1801487350325db" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8142e866b2d44844b1801487350325db" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 40061 } } }
I20260809 09:33:10.301859 1676 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8142e866b2d44844b1801487350325db [sys.catalog]: This master's current role is: LEADER
I20260809 09:33:10.304949 1681 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 09:33:10.305605 1681 catalog_manager.cc:1259] Loaded cluster ID: 38eff1d8bb6c4a8a98f1a2df53329d85
I20260809 09:33:10.305680 1681 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 09:33:10.308101 1681 catalog_manager.cc:1504] Loading token signing keys...
I20260809 09:33:10.308784 1681 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 8142e866b2d44844b1801487350325db: Loaded TSK: 0
I20260809 09:33:10.309990 1681 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260809 09:33:10.387584 1672 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 09:33:10.387887 1672 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 09:33:10.387965 1672 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 09:33:10.392069 1672 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 09:33:10.392155 1672 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 09:33:10.392246 1672 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.125.129
I20260809 09:33:10.397447 1672 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.125.148:32893
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786267941944493-20982-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786267941944493-20982-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786267941944493-20982-0/krb5kdc/oryx_127.20.125.129.keytab
--principal=oryx/127.20.125.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.125.129:36363
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786267941944493-20982-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.125.129
--webserver_port=42395
--tserver_master_addrs=127.20.125.190:40061
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1672
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.125.129
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786267941944493-20982-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 08:38:55 UTC on ccycloud.quasar-njjleu.root.comops.site
build id 4575935
I20260809 09:33:10.398768 1672 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 09:33:10.399834 1672 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 09:33:10.410941 1706 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:33:10.410951 1711 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:33:10.410950 1709 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:33:10.411871 1672 server_base.cc:1034] running on GCE node
I20260809 09:33:10.412289 1672 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 09:33:10.412966 1672 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 09:33:10.414203 1672 hybrid_clock.cc:648] HybridClock initialized: now 1786267990414155 us; error 48 us; skew 500 ppm
Aug 09 09:33:10 dist-test-slave-7kzw krb5kdc[799](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786267990, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.125.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 09:33:10.417387 1672 init.cc:377] Logged in from keytab as oryx/127.20.125.129@KRBTEST.COM (short username oryx)
I20260809 09:33:10.418418 1672 webserver.cc:460] Webserver started at http://127.20.125.129:42395/ using document root <none> and password file <none>
I20260809 09:33:10.419054 1672 fs_manager.cc:359] Metadata directory not provided
I20260809 09:33:10.419191 1672 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 09:33:10.423202 1672 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.003s
I20260809 09:33:10.425490 1724 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 09:33:10.426869 1672 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 09:33:10.426992 1672 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786267941944493-20982-0/minicluster-data/ts-0/data,/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786267941944493-20982-0/minicluster-data/ts-0/wal
uuid: "f9f2d5ca5c78445089aab1e5e8da693d"
format_stamp: "Formatted at 2026-08-09 09:33:09 on dist-test-slave-7kzw"
server_key: "18920d02f253dc3b5fa3616cf23cb4ab"
server_key_iv: "e093d14288406e8547ec1a298e4c17c7"
server_key_version: "encryptionkey@0"
I20260809 09:33:10.427541 1672 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786267941944493-20982-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786267941944493-20982-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786267941944493-20982-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 09:33:10.441710 1672 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 09:33:10.444795 1672 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 09:33:10.445007 1672 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 09:33:10.445619 1672 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 09:33:10.446539 1672 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 09:33:10.446589 1672 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 09:33:10.446660 1672 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 09:33:10.446692 1672 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 09:33:10.456059 1672 rpc_server.cc:307] RPC server started. Bound to: 127.20.125.129:36363
I20260809 09:33:10.456094 1887 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.125.129:36363 every 8 connection(s)
I20260809 09:33:10.457072 1672 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786267941944493-20982-0/minicluster-data/ts-0/data/info.pb
I20260809 09:33:10.462865 20982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu as pid 1672
I20260809 09:33:10.463439 20982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
/tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786267941944493-20982-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786267941944493-20982-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786267941944493-20982-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786267941944493-20982-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.125.130:35317
--local_ip_for_outbound_sockets=127.20.125.130
--tserver_master_addrs=127.20.125.190:40061
--webserver_port=46235
--webserver_interface=127.20.125.130
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.130.keytab
--principal=kudu/127.20.125.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.125.148:32893
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.20.125.130
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786267941944493-20982-0/krb5kdc/oryx_127.20.125.130.keytab with env {KRB5CCNAME=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786267941944493-20982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786267941944493-20982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786267941944493-20982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 09:33:10 dist-test-slave-7kzw krb5kdc[799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.125.129@KRBTEST.COM for oryx/127.20.125.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 09:33:10 dist-test-slave-7kzw krb5kdc[799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.125.129@KRBTEST.COM for oryx/127.20.125.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 09:33:10.470512 1891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:33:10.459388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:40061 (local address 127.20.125.129:37379)
0809 09:33:10.459873 (+ 485us) negotiation.cc:107] Waiting for socket to connect
0809 09:33:10.459933 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0809 09:33:10.461209 (+ 1276us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:33:10.462298 (+ 1089us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:33:10.462306 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:33:10.462779 (+ 473us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:33:10.463293 (+ 514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:33:10.463308 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:33:10.464659 (+ 1351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:33:10.464662 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:33:10.465528 (+ 866us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:33:10.465538 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:33:10.466309 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:33:10.466314 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:33:10.466557 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:33:10.467193 (+ 636us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 09:33:10.467211 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:33:10.469696 (+ 2485us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.125.190:40061: Server oryx/127.20.125.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":269,"thread_start_us":133,"threads_started":1}
W20260809 09:33:10.471024 1888 heartbeater.cc:644] Failed to heartbeat to 127.20.125.190:40061 (0 consecutive failures): Not authorized: Failed to ping master at 127.20.125.190:40061: Client connection negotiation failed: client connection to 127.20.125.190:40061: Server oryx/127.20.125.190@KRBTEST.COM not found in Kerberos database
W20260809 09:33:10.471009 1890 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 09:33:10.459267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.129:37379 (local address 127.20.125.190:40061)
0809 09:33:10.459821 (+ 554us) server_negotiation.cc:206] Beginning negotiation
0809 09:33:10.459841 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0809 09:33:10.460047 (+ 206us) server_negotiation.cc:407] Connection header received
0809 09:33:10.461616 (+ 1569us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:33:10.461626 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:33:10.461906 (+ 280us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:33:10.462228 (+ 322us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:33:10.463475 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:33:10.464515 (+ 1040us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:33:10.465662 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:33:10.466124 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:33:10.466304 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:33:10.470018 (+ 3714us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.125.129:37379: BlockingRecv error: recv got EOF from 127.20.125.129:37379 (error 108)
Metrics: {"server-negotiator.queue_time_us":373,"thread_start_us":135,"threads_started":1}
Aug 09 09:33:10 dist-test-slave-7kzw krb5kdc[799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.125.129@KRBTEST.COM for oryx/127.20.125.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 09:33:10 dist-test-slave-7kzw krb5kdc[799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.125.129@KRBTEST.COM for oryx/127.20.125.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 09:33:10.477277 1891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:33:10.471516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:40061 (local address 127.20.125.129:46921)
0809 09:33:10.471619 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0809 09:33:10.471640 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 09:33:10.471721 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:33:10.472019 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:33:10.472020 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:33:10.472153 (+ 133us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:33:10.472242 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:33:10.472247 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:33:10.473380 (+ 1133us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:33:10.473383 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:33:10.473910 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:33:10.473918 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:33:10.474582 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:33:10.474585 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:33:10.474651 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:33:10.475213 (+ 562us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 09:33:10.475223 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:33:10.477182 (+ 1959us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.125.190:40061: Server oryx/127.20.125.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":30}
W20260809 09:33:10.477464 1890 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 09:33:10.471583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.129:46921 (local address 127.20.125.190:40061)
0809 09:33:10.471734 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0809 09:33:10.471742 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 09:33:10.471782 (+ 40us) server_negotiation.cc:407] Connection header received
0809 09:33:10.471830 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:33:10.471836 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:33:10.471897 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:33:10.471991 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:33:10.472360 (+ 369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:33:10.473244 (+ 884us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:33:10.474027 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:33:10.474444 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:33:10.474597 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:33:10.477350 (+ 2753us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.125.129:46921: BlockingRecv error: recv got EOF from 127.20.125.129:46921 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Aug 09 09:33:10 dist-test-slave-7kzw krb5kdc[799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.125.129@KRBTEST.COM for oryx/127.20.125.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 09:33:10 dist-test-slave-7kzw krb5kdc[799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.125.129@KRBTEST.COM for oryx/127.20.125.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 09:33:10.482955 1891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:33:10.478000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:40061 (local address 127.20.125.129:52211)
0809 09:33:10.478182 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0809 09:33:10.478198 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 09:33:10.478276 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:33:10.478525 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:33:10.478527 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:33:10.478704 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:33:10.478773 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:33:10.478778 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:33:10.479488 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:33:10.479492 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:33:10.479997 (+ 505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:33:10.480005 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:33:10.480460 (+ 455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:33:10.480464 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:33:10.480549 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:33:10.481012 (+ 463us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 09:33:10.481023 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:33:10.482814 (+ 1791us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.125.190:40061: Server oryx/127.20.125.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":119}
W20260809 09:33:10.483050 1890 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 09:33:10.478122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.129:52211 (local address 127.20.125.190:40061)
0809 09:33:10.478262 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0809 09:33:10.478266 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 09:33:10.478280 (+ 14us) server_negotiation.cc:407] Connection header received
0809 09:33:10.478356 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:33:10.478359 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:33:10.478400 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:33:10.478520 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:33:10.478887 (+ 367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:33:10.479378 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:33:10.480118 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:33:10.480344 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:33:10.480449 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:33:10.482936 (+ 2487us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.125.129:52211: BlockingRecv error: recv got EOF from 127.20.125.129:52211 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
W20260809 09:33:10.483174 1888 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 09:33:10.583463 1892 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 09:33:10.583755 1892 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 09:33:10.583801 1892 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 09:33:10.587981 1892 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 09:33:10.588049 1892 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 09:33:10.588140 1892 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.125.130
I20260809 09:33:10.593575 1892 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.125.148:32893
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786267941944493-20982-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786267941944493-20982-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786267941944493-20982-0/krb5kdc/oryx_127.20.125.130.keytab
--principal=oryx/127.20.125.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.125.130:35317
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786267941944493-20982-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.125.130
--webserver_port=46235
--tserver_master_addrs=127.20.125.190:40061
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1892
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.125.130
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786267941944493-20982-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 08:38:55 UTC on ccycloud.quasar-njjleu.root.comops.site
build id 4575935
I20260809 09:33:10.594715 1892 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 09:33:10.595754 1892 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 09:33:10.605903 1902 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:33:10.606092 1892 server_base.cc:1034] running on GCE node
W20260809 09:33:10.605947 1900 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:33:10.605904 1898 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:33:10.606725 1892 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 09:33:10.607383 1892 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 09:33:10.608546 1892 hybrid_clock.cc:648] HybridClock initialized: now 1786267990608535 us; error 28 us; skew 500 ppm
Aug 09 09:33:10 dist-test-slave-7kzw krb5kdc[799](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786267990, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.125.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 09:33:10.611578 1892 init.cc:377] Logged in from keytab as oryx/127.20.125.130@KRBTEST.COM (short username oryx)
I20260809 09:33:10.612553 1892 webserver.cc:460] Webserver started at http://127.20.125.130:46235/ using document root <none> and password file <none>
I20260809 09:33:10.613101 1892 fs_manager.cc:359] Metadata directory not provided
I20260809 09:33:10.613184 1892 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 09:33:10.617261 1892 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260809 09:33:10.620023 1915 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 09:33:10.621541 1892 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 09:33:10.621657 1892 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786267941944493-20982-0/minicluster-data/ts-1/data,/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786267941944493-20982-0/minicluster-data/ts-1/wal
uuid: "97e14f4b033f480da0eace143d724235"
format_stamp: "Formatted at 2026-08-09 09:33:09 on dist-test-slave-7kzw"
server_key: "431f48e90f1c41a3be95971265a520ff"
server_key_iv: "45363598bfa32bd8500f0fd8ae7315aa"
server_key_version: "encryptionkey@0"
I20260809 09:33:10.622200 1892 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786267941944493-20982-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786267941944493-20982-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786267941944493-20982-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 09:33:10.637540 1892 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 09:33:10.641026 1892 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 09:33:10.641196 1892 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 09:33:10.641755 1892 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 09:33:10.642611 1892 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 09:33:10.642668 1892 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 09:33:10.642710 1892 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 09:33:10.642726 1892 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 09:33:10.652545 1892 rpc_server.cc:307] RPC server started. Bound to: 127.20.125.130:35317
I20260809 09:33:10.652594 2074 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.125.130:35317 every 8 connection(s)
I20260809 09:33:10.653595 1892 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786267941944493-20982-0/minicluster-data/ts-1/data/info.pb
I20260809 09:33:10.661546 20982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu as pid 1892
I20260809 09:33:10.662901 20982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
/tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786267941944493-20982-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786267941944493-20982-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786267941944493-20982-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786267941944493-20982-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.125.131:43633
--local_ip_for_outbound_sockets=127.20.125.131
--tserver_master_addrs=127.20.125.190:40061
--webserver_port=42597
--webserver_interface=127.20.125.131
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.131.keytab
--principal=kudu/127.20.125.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.125.148:32893
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.20.125.131
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786267941944493-20982-0/krb5kdc/oryx_127.20.125.131.keytab with env {KRB5CCNAME=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786267941944493-20982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786267941944493-20982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786267941944493-20982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 09:33:10 dist-test-slave-7kzw krb5kdc[799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.125.130@KRBTEST.COM for oryx/127.20.125.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 09:33:10 dist-test-slave-7kzw krb5kdc[799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.125.130@KRBTEST.COM for oryx/127.20.125.190@KRBTEST.COM, Server not found in Kerberos database
W20260809 09:33:10.666962 1890 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 09:33:10.655623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.130:40887 (local address 127.20.125.190:40061)
0809 09:33:10.655812 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0809 09:33:10.655816 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 09:33:10.656590 (+ 774us) server_negotiation.cc:407] Connection header received
0809 09:33:10.657985 (+ 1395us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:33:10.657988 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:33:10.658057 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:33:10.658162 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:33:10.659378 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:33:10.659909 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:33:10.661063 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:33:10.661478 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:33:10.661598 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:33:10.666833 (+ 5235us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.125.130:40887: BlockingRecv error: recv got EOF from 127.20.125.130:40887 (error 108)
Metrics: {"server-negotiator.queue_time_us":89}
I20260809 09:33:10.667970 2077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:33:10.655988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:40061 (local address 127.20.125.130:40887)
0809 09:33:10.656430 (+ 442us) negotiation.cc:107] Waiting for socket to connect
0809 09:33:10.656472 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 09:33:10.657764 (+ 1292us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:33:10.658353 (+ 589us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:33:10.658360 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:33:10.658739 (+ 379us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:33:10.659214 (+ 475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:33:10.659228 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:33:10.660039 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:33:10.660049 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:33:10.660906 (+ 857us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:33:10.660915 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:33:10.661767 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:33:10.661777 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:33:10.662031 (+ 254us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:33:10.662657 (+ 626us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 09:33:10.662692 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:33:10.666697 (+ 4005us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.125.190:40061: Server oryx/127.20.125.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":108,"threads_started":1}
W20260809 09:33:10.672667 2075 heartbeater.cc:644] Failed to heartbeat to 127.20.125.190:40061 (0 consecutive failures): Not authorized: Failed to ping master at 127.20.125.190:40061: Client connection negotiation failed: client connection to 127.20.125.190:40061: Server oryx/127.20.125.190@KRBTEST.COM not found in Kerberos database
Aug 09 09:33:10 dist-test-slave-7kzw krb5kdc[799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.125.130@KRBTEST.COM for oryx/127.20.125.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 09:33:10 dist-test-slave-7kzw krb5kdc[799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.125.130@KRBTEST.COM for oryx/127.20.125.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 09:33:10.682156 2077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:33:10.674719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:40061 (local address 127.20.125.130:51387)
0809 09:33:10.674903 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0809 09:33:10.674926 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0809 09:33:10.675043 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:33:10.675774 (+ 731us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:33:10.675777 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:33:10.675997 (+ 220us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:33:10.676156 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:33:10.676167 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:33:10.677398 (+ 1231us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:33:10.677402 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:33:10.678190 (+ 788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:33:10.678204 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:33:10.678859 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:33:10.678863 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:33:10.678976 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:33:10.679540 (+ 564us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 09:33:10.679557 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:33:10.681967 (+ 2410us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.125.190:40061: Server oryx/127.20.125.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":65}
W20260809 09:33:10.682241 1890 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 09:33:10.675035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.130:51387 (local address 127.20.125.190:40061)
0809 09:33:10.675521 (+ 486us) server_negotiation.cc:206] Beginning negotiation
0809 09:33:10.675524 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 09:33:10.675537 (+ 13us) server_negotiation.cc:407] Connection header received
0809 09:33:10.675579 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:33:10.675581 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:33:10.675654 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:33:10.675755 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:33:10.676319 (+ 564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:33:10.677235 (+ 916us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:33:10.678340 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:33:10.678730 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:33:10.678839 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:33:10.682113 (+ 3274us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.125.130:51387: BlockingRecv error: recv got EOF from 127.20.125.130:51387 (error 108)
Metrics: {"server-negotiator.queue_time_us":405}
Aug 09 09:33:10 dist-test-slave-7kzw krb5kdc[799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.125.130@KRBTEST.COM for oryx/127.20.125.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 09:33:10 dist-test-slave-7kzw krb5kdc[799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.125.130@KRBTEST.COM for oryx/127.20.125.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 09:33:10.689677 2077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:33:10.682895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:40061 (local address 127.20.125.130:50993)
0809 09:33:10.683084 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0809 09:33:10.683099 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 09:33:10.683199 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:33:10.683473 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:33:10.683478 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:33:10.683653 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:33:10.683764 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:33:10.683772 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:33:10.684801 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:33:10.684806 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:33:10.685924 (+ 1118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:33:10.685940 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:33:10.686847 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:33:10.686849 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:33:10.686908 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:33:10.687422 (+ 514us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 09:33:10.687444 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:33:10.689539 (+ 2095us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.125.190:40061: Server oryx/127.20.125.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":114}
W20260809 09:33:10.689786 1890 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 09:33:10.682955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.130:50993 (local address 127.20.125.190:40061)
0809 09:33:10.683073 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0809 09:33:10.683079 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 09:33:10.683160 (+ 81us) server_negotiation.cc:407] Connection header received
0809 09:33:10.683298 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:33:10.683301 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:33:10.683348 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:33:10.683429 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:33:10.683870 (+ 441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:33:10.684665 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:33:10.686098 (+ 1433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:33:10.686613 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:33:10.686696 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:33:10.689662 (+ 2966us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.125.130:50993: BlockingRecv error: recv got EOF from 127.20.125.130:50993 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
W20260809 09:33:10.690029 2075 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 09:33:10.790288 2078 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 09:33:10.790654 2078 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 09:33:10.790737 2078 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 09:33:10.794788 2078 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 09:33:10.794865 2078 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 09:33:10.794956 2078 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.125.131
I20260809 09:33:10.800202 2078 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.125.148:32893
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786267941944493-20982-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786267941944493-20982-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786267941944493-20982-0/krb5kdc/oryx_127.20.125.131.keytab
--principal=oryx/127.20.125.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.125.131:43633
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786267941944493-20982-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.125.131
--webserver_port=42597
--tserver_master_addrs=127.20.125.190:40061
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2078
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.125.131
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786267941944493-20982-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 08:38:55 UTC on ccycloud.quasar-njjleu.root.comops.site
build id 4575935
I20260809 09:33:10.801359 2078 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 09:33:10.802546 2078 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 09:33:10.812577 2088 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:33:10.812660 2090 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:33:10.812613 2086 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:33:10.812994 2078 server_base.cc:1034] running on GCE node
I20260809 09:33:10.813395 2078 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 09:33:10.814024 2078 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 09:33:10.815230 2078 hybrid_clock.cc:648] HybridClock initialized: now 1786267990815216 us; error 30 us; skew 500 ppm
Aug 09 09:33:10 dist-test-slave-7kzw krb5kdc[799](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786267990, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.125.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 09:33:10.818225 2078 init.cc:377] Logged in from keytab as oryx/127.20.125.131@KRBTEST.COM (short username oryx)
I20260809 09:33:10.819187 2078 webserver.cc:460] Webserver started at http://127.20.125.131:42597/ using document root <none> and password file <none>
I20260809 09:33:10.819718 2078 fs_manager.cc:359] Metadata directory not provided
I20260809 09:33:10.819793 2078 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 09:33:10.824018 2078 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260809 09:33:10.826221 2100 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 09:33:10.827514 2078 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 09:33:10.827636 2078 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786267941944493-20982-0/minicluster-data/ts-2/data,/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786267941944493-20982-0/minicluster-data/ts-2/wal
uuid: "7539421cfc5f47f3ab4e3290c87c9a23"
format_stamp: "Formatted at 2026-08-09 09:33:09 on dist-test-slave-7kzw"
server_key: "39b47792096b2e07cce3a7fa980d9e93"
server_key_iv: "bba5d4bf88fd871ba7fa1b1ab86e2bbb"
server_key_version: "encryptionkey@0"
I20260809 09:33:10.828140 2078 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786267941944493-20982-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786267941944493-20982-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786267941944493-20982-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 09:33:10.855715 2078 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 09:33:10.858927 2078 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 09:33:10.859088 2078 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 09:33:10.859653 2078 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 09:33:10.860450 2078 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 09:33:10.860496 2078 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 09:33:10.860538 2078 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 09:33:10.860553 2078 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 09:33:10.869850 2078 rpc_server.cc:307] RPC server started. Bound to: 127.20.125.131:43633
I20260809 09:33:10.869874 2259 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.125.131:43633 every 8 connection(s)
I20260809 09:33:10.870925 2078 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786267941944493-20982-0/minicluster-data/ts-2/data/info.pb
I20260809 09:33:10.873108 20982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu as pid 2078
Aug 09 09:33:10 dist-test-slave-7kzw krb5kdc[799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.125.131@KRBTEST.COM for oryx/127.20.125.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 09:33:10 dist-test-slave-7kzw krb5kdc[799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.125.131@KRBTEST.COM for oryx/127.20.125.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 09:33:10.884533 2265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:33:10.874247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38904 (local address 127.20.125.190:40061)
0809 09:33:10.874620 (+ 373us) server_negotiation.cc:206] Beginning negotiation
0809 09:33:10.874623 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 09:33:10.874635 (+ 12us) server_negotiation.cc:407] Connection header received
0809 09:33:10.874676 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:33:10.874678 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:33:10.874725 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:33:10.874805 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:33:10.875844 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:33:10.876545 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:33:10.877796 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:33:10.878184 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:33:10.878293 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:33:10.880027 (+ 1734us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:33:10.880033 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:33:10.880047 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:33:10.880081 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:33:10.882092 (+ 2011us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:33:10.882612 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:33:10.882621 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:33:10.882627 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:33:10.882798 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:33:10.883134 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:33:10.883138 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:33:10.883139 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:33:10.883604 (+ 465us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:33:10.883785 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0809 09:33:10.884093 (+ 308us) server_negotiation.cc:299] Negotiation successful
0809 09:33:10.884319 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":25,"server-negotiator.queue_time_us":302,"thread_start_us":87,"threads_started":1}
W20260809 09:33:10.884742 1890 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 09:33:10.872956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.131:42173 (local address 127.20.125.190:40061)
0809 09:33:10.873121 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0809 09:33:10.873125 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 09:33:10.874193 (+ 1068us) server_negotiation.cc:407] Connection header received
0809 09:33:10.876102 (+ 1909us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:33:10.876105 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:33:10.876152 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:33:10.876260 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:33:10.877796 (+ 1536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:33:10.878593 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:33:10.880269 (+ 1676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:33:10.880584 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:33:10.880694 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:33:10.884630 (+ 3936us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.125.131:42173: BlockingRecv error: recv got EOF from 127.20.125.131:42173 (error 108)
Metrics: {"mutex_wait_us":23,"server-negotiator.queue_time_us":65}
I20260809 09:33:10.885399 2262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:33:10.873399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:40061 (local address 127.20.125.131:42173)
0809 09:33:10.874038 (+ 639us) negotiation.cc:107] Waiting for socket to connect
0809 09:33:10.874093 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0809 09:33:10.875901 (+ 1808us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:33:10.876434 (+ 533us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:33:10.876440 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:33:10.877016 (+ 576us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:33:10.877646 (+ 630us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:33:10.877667 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:33:10.878738 (+ 1071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:33:10.878740 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:33:10.880153 (+ 1413us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:33:10.880163 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:33:10.880727 (+ 564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:33:10.880731 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:33:10.880884 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:33:10.881587 (+ 703us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 09:33:10.881602 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:33:10.884497 (+ 2895us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.125.190:40061: Server oryx/127.20.125.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":413,"thread_start_us":197,"threads_started":1}
W20260809 09:33:10.885874 2260 heartbeater.cc:644] Failed to heartbeat to 127.20.125.190:40061 (0 consecutive failures): Not authorized: Failed to ping master at 127.20.125.190:40061: Client connection negotiation failed: client connection to 127.20.125.190:40061: Server oryx/127.20.125.190@KRBTEST.COM not found in Kerberos database
Aug 09 09:33:10 dist-test-slave-7kzw krb5kdc[799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.125.131@KRBTEST.COM for oryx/127.20.125.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 09:33:10 dist-test-slave-7kzw krb5kdc[799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.125.131@KRBTEST.COM for oryx/127.20.125.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 09:33:10.891467 2262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:33:10.886272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:40061 (local address 127.20.125.131:45917)
0809 09:33:10.886394 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0809 09:33:10.886410 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 09:33:10.886508 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:33:10.886869 (+ 361us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:33:10.886872 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:33:10.887012 (+ 140us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:33:10.887104 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:33:10.887109 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:33:10.887997 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:33:10.888002 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:33:10.888538 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:33:10.888546 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:33:10.888997 (+ 451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:33:10.889001 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:33:10.889097 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:33:10.889574 (+ 477us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 09:33:10.889585 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:33:10.891320 (+ 1735us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.125.190:40061: Server oryx/127.20.125.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":50}
W20260809 09:33:10.891541 1890 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 09:33:10.886395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.131:45917 (local address 127.20.125.190:40061)
0809 09:33:10.886551 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0809 09:33:10.886554 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 09:33:10.886582 (+ 28us) server_negotiation.cc:407] Connection header received
0809 09:33:10.886618 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:33:10.886621 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:33:10.886672 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:33:10.886756 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:33:10.887226 (+ 470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:33:10.887888 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:33:10.888667 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:33:10.888890 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:33:10.888994 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:33:10.891428 (+ 2434us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.125.131:45917: BlockingRecv error: recv got EOF from 127.20.125.131:45917 (error 108)
Metrics: {"server-negotiator.queue_time_us":28}
Aug 09 09:33:10 dist-test-slave-7kzw krb5kdc[799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.125.131@KRBTEST.COM for oryx/127.20.125.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 09:33:10 dist-test-slave-7kzw krb5kdc[799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.125.131@KRBTEST.COM for oryx/127.20.125.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 09:33:10.896695 2262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:33:10.892025 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:40061 (local address 127.20.125.131:55627)
0809 09:33:10.892187 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0809 09:33:10.892202 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 09:33:10.892281 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:33:10.892503 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:33:10.892505 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:33:10.892652 (+ 147us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:33:10.892749 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:33:10.892754 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:33:10.893437 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:33:10.893442 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:33:10.893944 (+ 502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:33:10.893952 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:33:10.894400 (+ 448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:33:10.894403 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:33:10.894530 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:33:10.894974 (+ 444us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 09:33:10.894986 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:33:10.896545 (+ 1559us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.125.190:40061: Server oryx/127.20.125.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":43}
W20260809 09:33:10.896746 1890 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 09:33:10.892118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.131:55627 (local address 127.20.125.190:40061)
0809 09:33:10.892296 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0809 09:33:10.892301 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 09:33:10.892318 (+ 17us) server_negotiation.cc:407] Connection header received
0809 09:33:10.892366 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:33:10.892368 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:33:10.892409 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:33:10.892487 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:33:10.892861 (+ 374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:33:10.893334 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:33:10.894062 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:33:10.894280 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:33:10.894385 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:33:10.896646 (+ 2261us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.125.131:55627: BlockingRecv error: recv got EOF from 127.20.125.131:55627 (error 108)
Metrics: {"server-negotiator.queue_time_us":111}
W20260809 09:33:10.897011 2260 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 09 09:33:11 dist-test-slave-7kzw krb5kdc[799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.125.129@KRBTEST.COM for oryx/127.20.125.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 09:33:11 dist-test-slave-7kzw krb5kdc[799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.125.129@KRBTEST.COM for oryx/127.20.125.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 09:33:11.490509 2266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:33:11.483936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:40061 (local address 127.20.125.129:44359)
0809 09:33:11.484297 (+ 361us) negotiation.cc:107] Waiting for socket to connect
0809 09:33:11.484313 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 09:33:11.484426 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:33:11.484752 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:33:11.484754 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:33:11.484948 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:33:11.485052 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:33:11.485058 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:33:11.485961 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:33:11.485964 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:33:11.486661 (+ 697us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:33:11.486671 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:33:11.487148 (+ 477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:33:11.487151 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:33:11.487217 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:33:11.487825 (+ 608us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 09:33:11.487837 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:33:11.490321 (+ 2484us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.125.190:40061: Server oryx/127.20.125.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":273,"thread_start_us":118,"threads_started":1}
W20260809 09:33:11.490716 2267 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 09:33:11.484003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.129:44359 (local address 127.20.125.190:40061)
0809 09:33:11.484318 (+ 315us) server_negotiation.cc:206] Beginning negotiation
0809 09:33:11.484325 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 09:33:11.484395 (+ 70us) server_negotiation.cc:407] Connection header received
0809 09:33:11.484524 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:33:11.484527 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:33:11.484589 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:33:11.484673 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:33:11.485226 (+ 553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:33:11.485833 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:33:11.486798 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:33:11.487030 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:33:11.487119 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:33:11.490523 (+ 3404us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.125.129:44359: BlockingRecv error: recv got EOF from 127.20.125.129:44359 (error 108)
Metrics: {"server-negotiator.queue_time_us":221,"thread_start_us":125,"threads_started":1}
Aug 09 09:33:11 dist-test-slave-7kzw krb5kdc[799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.125.130@KRBTEST.COM for oryx/127.20.125.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 09:33:11 dist-test-slave-7kzw krb5kdc[799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.125.130@KRBTEST.COM for oryx/127.20.125.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 09:33:11.697209 2269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:33:11.690722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:40061 (local address 127.20.125.130:47201)
0809 09:33:11.691021 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0809 09:33:11.691036 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 09:33:11.691184 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:33:11.691560 (+ 376us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:33:11.691565 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:33:11.691849 (+ 284us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:33:11.691964 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:33:11.691970 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:33:11.692975 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:33:11.692978 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:33:11.693603 (+ 625us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:33:11.693612 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:33:11.694160 (+ 548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:33:11.694163 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:33:11.694246 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:33:11.694902 (+ 656us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 09:33:11.694925 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:33:11.697062 (+ 2137us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.125.190:40061: Server oryx/127.20.125.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":104,"threads_started":1}
W20260809 09:33:11.697352 2267 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 09:33:11.690895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.130:47201 (local address 127.20.125.190:40061)
0809 09:33:11.691042 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0809 09:33:11.691049 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 09:33:11.691150 (+ 101us) server_negotiation.cc:407] Connection header received
0809 09:33:11.691342 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:33:11.691346 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:33:11.691407 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:33:11.691516 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:33:11.692101 (+ 585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:33:11.692862 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:33:11.693784 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:33:11.694038 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:33:11.694137 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:33:11.697221 (+ 3084us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.125.130:47201: BlockingRecv error: recv got EOF from 127.20.125.130:47201 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Aug 09 09:33:11 dist-test-slave-7kzw krb5kdc[799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.125.131@KRBTEST.COM for oryx/127.20.125.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 09:33:11 dist-test-slave-7kzw krb5kdc[799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.125.131@KRBTEST.COM for oryx/127.20.125.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 09:33:11.904166 2270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:33:11.897851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:40061 (local address 127.20.125.131:60715)
0809 09:33:11.898208 (+ 357us) negotiation.cc:107] Waiting for socket to connect
0809 09:33:11.898228 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 09:33:11.898359 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:33:11.898705 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:33:11.898708 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:33:11.898939 (+ 231us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:33:11.899062 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:33:11.899068 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:33:11.899933 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:33:11.899939 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:33:11.900579 (+ 640us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:33:11.900588 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:33:11.901192 (+ 604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:33:11.901196 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:33:11.901306 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:33:11.901924 (+ 618us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 09:33:11.901939 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:33:11.904028 (+ 2089us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.125.190:40061: Server oryx/127.20.125.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":274,"thread_start_us":91,"threads_started":1}
W20260809 09:33:11.904368 2267 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 09:33:11.898002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.131:60715 (local address 127.20.125.190:40061)
0809 09:33:11.898188 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0809 09:33:11.898219 (+ 31us) server_negotiation.cc:399] Waiting for connection header
0809 09:33:11.898297 (+ 78us) server_negotiation.cc:407] Connection header received
0809 09:33:11.898507 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:33:11.898510 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:33:11.898572 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:33:11.898673 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:33:11.899201 (+ 528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:33:11.899806 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:33:11.900706 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:33:11.901055 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:33:11.901179 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:33:11.904209 (+ 3030us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.125.131:60715: BlockingRecv error: recv got EOF from 127.20.125.131:60715 (error 108)
Metrics: {"server-negotiator.queue_time_us":85}
Aug 09 09:33:12 dist-test-slave-7kzw krb5kdc[799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.125.129@KRBTEST.COM for oryx/127.20.125.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 09:33:12 dist-test-slave-7kzw krb5kdc[799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.125.129@KRBTEST.COM for oryx/127.20.125.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 09:33:12.501143 2271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:33:12.491374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:40061 (local address 127.20.125.129:60035)
0809 09:33:12.491660 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0809 09:33:12.491675 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 09:33:12.491769 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:33:12.492021 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:33:12.492024 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:33:12.492227 (+ 203us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:33:12.492327 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:33:12.492332 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:33:12.493130 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:33:12.493133 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:33:12.493797 (+ 664us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:33:12.493808 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:33:12.494272 (+ 464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:33:12.494275 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:33:12.494344 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:33:12.498685 (+ 4341us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 09:33:12.498698 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:33:12.500965 (+ 2267us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.125.190:40061: Server oryx/127.20.125.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":91,"threads_started":1}
W20260809 09:33:12.501219 2272 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 09:33:12.491434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.129:60035 (local address 127.20.125.190:40061)
0809 09:33:12.491708 (+ 274us) server_negotiation.cc:206] Beginning negotiation
0809 09:33:12.491711 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 09:33:12.491725 (+ 14us) server_negotiation.cc:407] Connection header received
0809 09:33:12.491858 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:33:12.491860 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:33:12.491906 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:33:12.491983 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:33:12.492434 (+ 451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:33:12.492971 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:33:12.493927 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:33:12.494149 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:33:12.494242 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:33:12.501108 (+ 6866us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.125.129:60035: BlockingRecv error: recv got EOF from 127.20.125.129:60035 (error 108)
Metrics: {"server-negotiator.queue_time_us":194,"thread_start_us":93,"threads_started":1}
Aug 09 09:33:12 dist-test-slave-7kzw krb5kdc[799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.125.130@KRBTEST.COM for oryx/127.20.125.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 09:33:12 dist-test-slave-7kzw krb5kdc[799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.125.130@KRBTEST.COM for oryx/127.20.125.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 09:33:12.704111 2273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:33:12.698151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:40061 (local address 127.20.125.130:50031)
0809 09:33:12.698446 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0809 09:33:12.698510 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0809 09:33:12.698653 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:33:12.698944 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:33:12.698946 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:33:12.699167 (+ 221us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:33:12.699269 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:33:12.699275 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:33:12.700042 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:33:12.700047 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:33:12.700610 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:33:12.700619 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:33:12.701169 (+ 550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:33:12.701172 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:33:12.701266 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:33:12.701747 (+ 481us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 09:33:12.701759 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:33:12.703914 (+ 2155us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.125.190:40061: Server oryx/127.20.125.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":90,"threads_started":1}
W20260809 09:33:12.704162 2272 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 09:33:12.698254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.130:50031 (local address 127.20.125.190:40061)
0809 09:33:12.698416 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0809 09:33:12.698421 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 09:33:12.698587 (+ 166us) server_negotiation.cc:407] Connection header received
0809 09:33:12.698764 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:33:12.698768 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:33:12.698830 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:33:12.698916 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:33:12.699389 (+ 473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:33:12.699929 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:33:12.700747 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:33:12.701042 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:33:12.701155 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:33:12.704052 (+ 2897us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.125.130:50031: BlockingRecv error: recv got EOF from 127.20.125.130:50031 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
Aug 09 09:33:12 dist-test-slave-7kzw krb5kdc[799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.125.131@KRBTEST.COM for oryx/127.20.125.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 09:33:12 dist-test-slave-7kzw krb5kdc[799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.125.131@KRBTEST.COM for oryx/127.20.125.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 09:33:12.911362 2274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:33:12.905071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:40061 (local address 127.20.125.131:37331)
0809 09:33:12.905445 (+ 374us) negotiation.cc:107] Waiting for socket to connect
0809 09:33:12.905460 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 09:33:12.905569 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:33:12.905871 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:33:12.905873 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:33:12.906144 (+ 271us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:33:12.906250 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:33:12.906255 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:33:12.907151 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:33:12.907154 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:33:12.907770 (+ 616us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:33:12.907779 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:33:12.908332 (+ 553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:33:12.908335 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:33:12.908424 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:33:12.909029 (+ 605us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 09:33:12.909042 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:33:12.911154 (+ 2112us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.125.190:40061: Server oryx/127.20.125.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":272,"thread_start_us":118,"threads_started":1}
W20260809 09:33:12.911458 2272 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 09:33:12.905286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.131:37331 (local address 127.20.125.190:40061)
0809 09:33:12.905476 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0809 09:33:12.905480 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 09:33:12.905518 (+ 38us) server_negotiation.cc:407] Connection header received
0809 09:33:12.905663 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:33:12.905666 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:33:12.905748 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:33:12.905819 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:33:12.906397 (+ 578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:33:12.907000 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:33:12.907920 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:33:12.908195 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:33:12.908319 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:33:12.911306 (+ 2987us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.125.131:37331: BlockingRecv error: recv got EOF from 127.20.125.131:37331 (error 108)
Metrics: {"server-negotiator.queue_time_us":94}
Aug 09 09:33:13 dist-test-slave-7kzw krb5kdc[799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.125.129@KRBTEST.COM for oryx/127.20.125.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 09:33:13 dist-test-slave-7kzw krb5kdc[799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.125.129@KRBTEST.COM for oryx/127.20.125.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 09:33:13.507797 2275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:33:13.502123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:40061 (local address 127.20.125.129:39379)
0809 09:33:13.502431 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0809 09:33:13.502445 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 09:33:13.502600 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:33:13.502843 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:33:13.502845 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:33:13.503032 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:33:13.503129 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:33:13.503134 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:33:13.503941 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:33:13.503944 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:33:13.504489 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:33:13.504497 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:33:13.504932 (+ 435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:33:13.504936 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:33:13.505040 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:33:13.505634 (+ 594us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 09:33:13.505646 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:33:13.507657 (+ 2011us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.125.190:40061: Server oryx/127.20.125.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":100,"threads_started":1}
W20260809 09:33:13.507967 2276 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 09:33:13.502161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.129:39379 (local address 127.20.125.190:40061)
0809 09:33:13.502453 (+ 292us) server_negotiation.cc:206] Beginning negotiation
0809 09:33:13.502457 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 09:33:13.502547 (+ 90us) server_negotiation.cc:407] Connection header received
0809 09:33:13.502674 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:33:13.502676 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:33:13.502724 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:33:13.502810 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:33:13.503267 (+ 457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:33:13.503831 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:33:13.504592 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:33:13.504813 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:33:13.504893 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:33:13.507797 (+ 2904us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.125.129:39379: BlockingRecv error: recv got EOF from 127.20.125.129:39379 (error 108)
Metrics: {"server-negotiator.queue_time_us":208,"thread_start_us":91,"threads_started":1}
Aug 09 09:33:13 dist-test-slave-7kzw krb5kdc[799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.125.130@KRBTEST.COM for oryx/127.20.125.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 09:33:13 dist-test-slave-7kzw krb5kdc[799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.125.130@KRBTEST.COM for oryx/127.20.125.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 09:33:13.711520 2277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:33:13.705119 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:40061 (local address 127.20.125.130:55065)
0809 09:33:13.705431 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0809 09:33:13.705446 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 09:33:13.705598 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:33:13.705913 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:33:13.705917 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:33:13.706150 (+ 233us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:33:13.706259 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:33:13.706266 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:33:13.707188 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:33:13.707195 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:33:13.707826 (+ 631us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:33:13.707835 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:33:13.708374 (+ 539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:33:13.708378 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:33:13.708476 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:33:13.709112 (+ 636us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 09:33:13.709125 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:33:13.711309 (+ 2184us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.125.190:40061: Server oryx/127.20.125.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":106,"threads_started":1}
W20260809 09:33:13.711582 2276 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 09:33:13.705264 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.130:55065 (local address 127.20.125.190:40061)
0809 09:33:13.705412 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0809 09:33:13.705416 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 09:33:13.705538 (+ 122us) server_negotiation.cc:407] Connection header received
0809 09:33:13.705719 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:33:13.705723 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:33:13.705785 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:33:13.705884 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:33:13.706395 (+ 511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:33:13.707063 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:33:13.707965 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:33:13.708242 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:33:13.708369 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:33:13.711458 (+ 3089us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.125.130:55065: BlockingRecv error: recv got EOF from 127.20.125.130:55065 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Aug 09 09:33:13 dist-test-slave-7kzw krb5kdc[799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.125.131@KRBTEST.COM for oryx/127.20.125.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 09:33:13 dist-test-slave-7kzw krb5kdc[799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.125.131@KRBTEST.COM for oryx/127.20.125.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 09:33:13.918818 2278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:33:13.912264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:40061 (local address 127.20.125.131:32941)
0809 09:33:13.912558 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0809 09:33:13.912573 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 09:33:13.912681 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:33:13.913002 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:33:13.913005 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:33:13.913212 (+ 207us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:33:13.913316 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:33:13.913322 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:33:13.914199 (+ 877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:33:13.914201 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:33:13.914915 (+ 714us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:33:13.914926 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:33:13.915398 (+ 472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:33:13.915401 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:33:13.915468 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:33:13.916089 (+ 621us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 09:33:13.916102 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:33:13.918698 (+ 2596us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.125.190:40061: Server oryx/127.20.125.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":103,"threads_started":1}
W20260809 09:33:13.919032 2276 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 09:33:13.912328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.131:32941 (local address 127.20.125.190:40061)
0809 09:33:13.912518 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0809 09:33:13.912522 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 09:33:13.912654 (+ 132us) server_negotiation.cc:407] Connection header received
0809 09:33:13.912817 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:33:13.912820 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:33:13.912875 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:33:13.912958 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:33:13.913469 (+ 511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:33:13.914087 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:33:13.915039 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:33:13.915287 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:33:13.915375 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:33:13.918852 (+ 3477us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.125.131:32941: BlockingRecv error: recv got EOF from 127.20.125.131:32941 (error 108)
Metrics: {"server-negotiator.queue_time_us":91}
Aug 09 09:33:14 dist-test-slave-7kzw krb5kdc[799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.125.129@KRBTEST.COM for oryx/127.20.125.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 09:33:14 dist-test-slave-7kzw krb5kdc[799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.125.129@KRBTEST.COM for oryx/127.20.125.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 09:33:14.514828 2280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:33:14.508591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:40061 (local address 127.20.125.129:36787)
0809 09:33:14.508879 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0809 09:33:14.508893 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 09:33:14.509004 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:33:14.509260 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:33:14.509262 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:33:14.509451 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:33:14.509550 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:33:14.509555 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:33:14.510720 (+ 1165us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:33:14.510725 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:33:14.511292 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:33:14.511301 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:33:14.511806 (+ 505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:33:14.511809 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:33:14.511880 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:33:14.512492 (+ 612us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 09:33:14.512504 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:33:14.514648 (+ 2144us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.125.190:40061: Server oryx/127.20.125.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":95,"threads_started":1}
W20260809 09:33:14.514930 2282 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 09:33:14.508649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.129:36787 (local address 127.20.125.190:40061)
0809 09:33:14.508959 (+ 310us) server_negotiation.cc:206] Beginning negotiation
0809 09:33:14.508966 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 09:33:14.508982 (+ 16us) server_negotiation.cc:407] Connection header received
0809 09:33:14.509090 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:33:14.509092 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:33:14.509141 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:33:14.509222 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:33:14.509692 (+ 470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:33:14.510350 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:33:14.511411 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:33:14.511653 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:33:14.511737 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:33:14.514790 (+ 3053us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.125.129:36787: BlockingRecv error: recv got EOF from 127.20.125.129:36787 (error 108)
Metrics: {"server-negotiator.queue_time_us":212,"thread_start_us":101,"threads_started":1}
Aug 09 09:33:14 dist-test-slave-7kzw krb5kdc[799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.125.130@KRBTEST.COM for oryx/127.20.125.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 09:33:14 dist-test-slave-7kzw krb5kdc[799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.125.130@KRBTEST.COM for oryx/127.20.125.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 09:33:14.719203 2283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:33:14.712446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:40061 (local address 127.20.125.130:38249)
0809 09:33:14.712863 (+ 417us) negotiation.cc:107] Waiting for socket to connect
0809 09:33:14.712876 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 09:33:14.713018 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:33:14.713328 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:33:14.713331 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:33:14.713550 (+ 219us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:33:14.713691 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:33:14.713701 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:33:14.714905 (+ 1204us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:33:14.714908 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:33:14.715613 (+ 705us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:33:14.715623 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:33:14.716077 (+ 454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:33:14.716080 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:33:14.716147 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:33:14.716811 (+ 664us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 09:33:14.716824 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:33:14.719067 (+ 2243us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.125.190:40061: Server oryx/127.20.125.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":308,"thread_start_us":114,"threads_started":1}
W20260809 09:33:14.719353 2282 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 09:33:14.712590 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.130:38249 (local address 127.20.125.190:40061)
0809 09:33:14.712794 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0809 09:33:14.712800 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 09:33:14.712953 (+ 153us) server_negotiation.cc:407] Connection header received
0809 09:33:14.713152 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:33:14.713156 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:33:14.713211 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:33:14.713302 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:33:14.713887 (+ 585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:33:14.714718 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:33:14.715730 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:33:14.715949 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:33:14.716032 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:33:14.719230 (+ 3198us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.125.130:38249: BlockingRecv error: recv got EOF from 127.20.125.130:38249 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Aug 09 09:33:14 dist-test-slave-7kzw krb5kdc[799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.125.131@KRBTEST.COM for oryx/127.20.125.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 09:33:14 dist-test-slave-7kzw krb5kdc[799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.125.131@KRBTEST.COM for oryx/127.20.125.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 09:33:14.926178 2284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:33:14.919621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:40061 (local address 127.20.125.131:51839)
0809 09:33:14.920021 (+ 400us) negotiation.cc:107] Waiting for socket to connect
0809 09:33:14.920038 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 09:33:14.920201 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:33:14.920513 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:33:14.920516 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:33:14.920764 (+ 248us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:33:14.920867 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:33:14.920874 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:33:14.921727 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:33:14.921734 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:33:14.922323 (+ 589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:33:14.922331 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:33:14.922855 (+ 524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:33:14.922859 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:33:14.922955 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:33:14.923606 (+ 651us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 09:33:14.923643 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:33:14.925935 (+ 2292us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.125.190:40061: Server oryx/127.20.125.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":319,"thread_start_us":113,"threads_started":1}
W20260809 09:33:14.926198 2282 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 09:33:14.919787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.131:51839 (local address 127.20.125.190:40061)
0809 09:33:14.919984 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0809 09:33:14.919989 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 09:33:14.920140 (+ 151us) server_negotiation.cc:407] Connection header received
0809 09:33:14.920323 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:33:14.920327 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:33:14.920389 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:33:14.920486 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:33:14.921013 (+ 527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:33:14.921600 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:33:14.922484 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:33:14.922728 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:33:14.922855 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:33:14.926061 (+ 3206us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.125.131:51839: BlockingRecv error: recv got EOF from 127.20.125.131:51839 (error 108)
Metrics: {"server-negotiator.queue_time_us":88}
Aug 09 09:33:15 dist-test-slave-7kzw krb5kdc[799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.125.129@KRBTEST.COM for oryx/127.20.125.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 09:33:15 dist-test-slave-7kzw krb5kdc[799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.125.129@KRBTEST.COM for oryx/127.20.125.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 09:33:15.521287 2285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:33:15.515610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:40061 (local address 127.20.125.129:57673)
0809 09:33:15.515926 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0809 09:33:15.515941 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 09:33:15.516040 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:33:15.516321 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:33:15.516323 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:33:15.516528 (+ 205us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:33:15.516661 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:33:15.516667 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:33:15.517453 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:33:15.517455 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:33:15.518050 (+ 595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:33:15.518059 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:33:15.518565 (+ 506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:33:15.518568 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:33:15.518663 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:33:15.519156 (+ 493us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 09:33:15.519169 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:33:15.521153 (+ 1984us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.125.190:40061: Server oryx/127.20.125.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":232,"thread_start_us":93,"threads_started":1}
W20260809 09:33:15.521469 2286 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 09:33:15.515733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.129:57673 (local address 127.20.125.190:40061)
0809 09:33:15.515999 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0809 09:33:15.516003 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 09:33:15.516017 (+ 14us) server_negotiation.cc:407] Connection header received
0809 09:33:15.516156 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:33:15.516158 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:33:15.516204 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:33:15.516281 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:33:15.516799 (+ 518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:33:15.517349 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:33:15.518174 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:33:15.518407 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:33:15.518511 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:33:15.521319 (+ 2808us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.125.129:57673: BlockingRecv error: recv got EOF from 127.20.125.129:57673 (error 108)
Metrics: {"server-negotiator.queue_time_us":197,"thread_start_us":88,"threads_started":1}
Aug 09 09:33:15 dist-test-slave-7kzw krb5kdc[799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.125.130@KRBTEST.COM for oryx/127.20.125.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 09:33:15 dist-test-slave-7kzw krb5kdc[799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.125.130@KRBTEST.COM for oryx/127.20.125.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 09:33:15.726094 2287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:33:15.720081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:40061 (local address 127.20.125.130:54461)
0809 09:33:15.720395 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0809 09:33:15.720409 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 09:33:15.720492 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:33:15.720784 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:33:15.720787 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:33:15.720988 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:33:15.721098 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:33:15.721104 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:33:15.721871 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:33:15.721874 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:33:15.722501 (+ 627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:33:15.722510 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:33:15.722997 (+ 487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:33:15.723000 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:33:15.723066 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:33:15.723775 (+ 709us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 09:33:15.723788 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:33:15.725973 (+ 2185us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.125.190:40061: Server oryx/127.20.125.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":116,"threads_started":1}
W20260809 09:33:15.726318 2286 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 09:33:15.720267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.130:54461 (local address 127.20.125.190:40061)
0809 09:33:15.720434 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0809 09:33:15.720438 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 09:33:15.720453 (+ 15us) server_negotiation.cc:407] Connection header received
0809 09:33:15.720582 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:33:15.720584 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:33:15.720630 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:33:15.720705 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:33:15.721210 (+ 505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:33:15.721742 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:33:15.722655 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:33:15.722882 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:33:15.722974 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:33:15.726189 (+ 3215us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.125.130:54461: BlockingRecv error: recv got EOF from 127.20.125.130:54461 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
Aug 09 09:33:15 dist-test-slave-7kzw krb5kdc[799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.125.131@KRBTEST.COM for oryx/127.20.125.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 09:33:15 dist-test-slave-7kzw krb5kdc[799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.125.131@KRBTEST.COM for oryx/127.20.125.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 09:33:15.933346 2288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:33:15.927043 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:40061 (local address 127.20.125.131:33793)
0809 09:33:15.927389 (+ 346us) negotiation.cc:107] Waiting for socket to connect
0809 09:33:15.927403 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 09:33:15.927502 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:33:15.927854 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:33:15.927857 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:33:15.928083 (+ 226us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:33:15.928192 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:33:15.928199 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:33:15.929018 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:33:15.929020 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:33:15.929611 (+ 591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:33:15.929625 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:33:15.930092 (+ 467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:33:15.930096 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:33:15.930173 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:33:15.930980 (+ 807us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 09:33:15.930993 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:33:15.933152 (+ 2159us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.125.190:40061: Server oryx/127.20.125.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":270,"thread_start_us":97,"threads_started":1}
W20260809 09:33:15.933374 2286 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 09:33:15.927155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.131:33793 (local address 127.20.125.190:40061)
0809 09:33:15.927299 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0809 09:33:15.927303 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 09:33:15.927510 (+ 207us) server_negotiation.cc:407] Connection header received
0809 09:33:15.927567 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:33:15.927569 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:33:15.927661 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:33:15.927746 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:33:15.928351 (+ 605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:33:15.928892 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:33:15.929737 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:33:15.929961 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:33:15.930051 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:33:15.933267 (+ 3216us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.125.131:33793: BlockingRecv error: recv got EOF from 127.20.125.131:33793 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Aug 09 09:33:16 dist-test-slave-7kzw krb5kdc[799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.125.129@KRBTEST.COM for oryx/127.20.125.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 09:33:16 dist-test-slave-7kzw krb5kdc[799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.125.129@KRBTEST.COM for oryx/127.20.125.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 09:33:16.528410 2289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:33:16.522410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:40061 (local address 127.20.125.129:55211)
0809 09:33:16.522922 (+ 512us) negotiation.cc:107] Waiting for socket to connect
0809 09:33:16.522936 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 09:33:16.523034 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:33:16.523272 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:33:16.523274 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:33:16.523470 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:33:16.523567 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:33:16.523572 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:33:16.524423 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:33:16.524427 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:33:16.525073 (+ 646us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:33:16.525083 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:33:16.525609 (+ 526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:33:16.525612 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:33:16.525731 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:33:16.526275 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 09:33:16.526287 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:33:16.528294 (+ 2007us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.125.190:40061: Server oryx/127.20.125.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":428,"thread_start_us":171,"threads_started":1}
W20260809 09:33:16.528630 2293 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 09:33:16.522490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.129:55211 (local address 127.20.125.190:40061)
0809 09:33:16.522802 (+ 312us) server_negotiation.cc:206] Beginning negotiation
0809 09:33:16.522826 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0809 09:33:16.523008 (+ 182us) server_negotiation.cc:407] Connection header received
0809 09:33:16.523106 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:33:16.523108 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:33:16.523155 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:33:16.523231 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:33:16.523718 (+ 487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:33:16.524308 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:33:16.525231 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:33:16.525497 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:33:16.525698 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:33:16.528441 (+ 2743us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.125.129:55211: BlockingRecv error: recv got EOF from 127.20.125.129:55211 (error 108)
Metrics: {"server-negotiator.queue_time_us":224,"thread_start_us":94,"threads_started":1}
Aug 09 09:33:16 dist-test-slave-7kzw krb5kdc[799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.125.130@KRBTEST.COM for oryx/127.20.125.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 09:33:16 dist-test-slave-7kzw krb5kdc[799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.125.130@KRBTEST.COM for oryx/127.20.125.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 09:33:16.733450 2294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:33:16.726907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:40061 (local address 127.20.125.130:56625)
0809 09:33:16.727379 (+ 472us) negotiation.cc:107] Waiting for socket to connect
0809 09:33:16.727398 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 09:33:16.727566 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:33:16.727926 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:33:16.727934 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:33:16.728200 (+ 266us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:33:16.728304 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:33:16.728310 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:33:16.729183 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:33:16.729189 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:33:16.729804 (+ 615us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:33:16.729812 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:33:16.730332 (+ 520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:33:16.730336 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:33:16.730422 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:33:16.731076 (+ 654us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 09:33:16.731094 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:33:16.733331 (+ 2237us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.125.190:40061: Server oryx/127.20.125.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":391,"thread_start_us":96,"threads_started":1}
W20260809 09:33:16.733652 2293 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 09:33:16.727046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.130:56625 (local address 127.20.125.190:40061)
0809 09:33:16.727371 (+ 325us) server_negotiation.cc:206] Beginning negotiation
0809 09:33:16.727377 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 09:33:16.727497 (+ 120us) server_negotiation.cc:407] Connection header received
0809 09:33:16.727717 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:33:16.727721 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:33:16.727790 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:33:16.727891 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:33:16.728444 (+ 553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:33:16.729051 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:33:16.729946 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:33:16.730202 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:33:16.730326 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:33:16.733499 (+ 3173us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.125.130:56625: BlockingRecv error: recv got EOF from 127.20.125.130:56625 (error 108)
Metrics: {"server-negotiator.queue_time_us":179}
Aug 09 09:33:16 dist-test-slave-7kzw krb5kdc[799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.125.131@KRBTEST.COM for oryx/127.20.125.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 09:33:16 dist-test-slave-7kzw krb5kdc[799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.125.131@KRBTEST.COM for oryx/127.20.125.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 09:33:16.940057 2295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:33:16.934250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:40061 (local address 127.20.125.131:50675)
0809 09:33:16.934547 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0809 09:33:16.934562 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 09:33:16.934680 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:33:16.934928 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:33:16.934930 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:33:16.935140 (+ 210us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:33:16.935236 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:33:16.935243 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:33:16.936000 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:33:16.936003 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:33:16.936562 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:33:16.936572 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:33:16.937172 (+ 600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:33:16.937176 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:33:16.937266 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:33:16.937849 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 09:33:16.937861 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:33:16.939877 (+ 2016us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.125.190:40061: Server oryx/127.20.125.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":74,"threads_started":1}
W20260809 09:33:16.940112 2293 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 09:33:16.934367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.131:50675 (local address 127.20.125.190:40061)
0809 09:33:16.934563 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0809 09:33:16.934568 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 09:33:16.934639 (+ 71us) server_negotiation.cc:407] Connection header received
0809 09:33:16.934766 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:33:16.934768 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:33:16.934818 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:33:16.934885 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:33:16.935339 (+ 454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:33:16.935875 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:33:16.936711 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:33:16.936970 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:33:16.937086 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:33:16.940003 (+ 2917us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.125.131:50675: BlockingRecv error: recv got EOF from 127.20.125.131:50675 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
Aug 09 09:33:17 dist-test-slave-7kzw krb5kdc[799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.125.129@KRBTEST.COM for oryx/127.20.125.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 09:33:17 dist-test-slave-7kzw krb5kdc[799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.125.129@KRBTEST.COM for oryx/127.20.125.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 09:33:17.539362 2296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:33:17.529220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:40061 (local address 127.20.125.129:56659)
0809 09:33:17.529563 (+ 343us) negotiation.cc:107] Waiting for socket to connect
0809 09:33:17.529577 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 09:33:17.529676 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:33:17.529909 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:33:17.529910 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:33:17.530102 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:33:17.530196 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:33:17.530202 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:33:17.530957 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:33:17.530959 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:33:17.531545 (+ 586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:33:17.531555 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:33:17.532161 (+ 606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:33:17.532167 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:33:17.532260 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:33:17.537109 (+ 4849us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 09:33:17.537122 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:33:17.539179 (+ 2057us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.125.190:40061: Server oryx/127.20.125.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":250,"thread_start_us":98,"threads_started":1}
W20260809 09:33:17.539489 2297 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 09:33:17.529310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.129:56659 (local address 127.20.125.190:40061)
0809 09:33:17.529571 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0809 09:33:17.529574 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 09:33:17.529649 (+ 75us) server_negotiation.cc:407] Connection header received
0809 09:33:17.529772 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:33:17.529774 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:33:17.529821 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:33:17.529897 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:33:17.530304 (+ 407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:33:17.530859 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:33:17.531732 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:33:17.532010 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:33:17.532166 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:33:17.539335 (+ 7169us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.125.129:56659: BlockingRecv error: recv got EOF from 127.20.125.129:56659 (error 108)
Metrics: {"server-negotiator.queue_time_us":185,"thread_start_us":108,"threads_started":1}
Aug 09 09:33:17 dist-test-slave-7kzw krb5kdc[799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.125.130@KRBTEST.COM for oryx/127.20.125.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 09:33:17 dist-test-slave-7kzw krb5kdc[799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.125.130@KRBTEST.COM for oryx/127.20.125.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 09:33:17.740590 2301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:33:17.734272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:40061 (local address 127.20.125.130:54553)
0809 09:33:17.734632 (+ 360us) negotiation.cc:107] Waiting for socket to connect
0809 09:33:17.734646 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 09:33:17.734751 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:33:17.735071 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:33:17.735082 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:33:17.735282 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:33:17.735385 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:33:17.735390 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:33:17.736142 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:33:17.736145 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:33:17.736763 (+ 618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:33:17.736773 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:33:17.737426 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:33:17.737431 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:33:17.737585 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:33:17.738307 (+ 722us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 09:33:17.738323 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:33:17.740468 (+ 2145us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.125.190:40061: Server oryx/127.20.125.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":278,"thread_start_us":112,"threads_started":1}
W20260809 09:33:17.740770 2297 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 09:33:17.734343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.130:54553 (local address 127.20.125.190:40061)
0809 09:33:17.734641 (+ 298us) server_negotiation.cc:206] Beginning negotiation
0809 09:33:17.734645 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 09:33:17.734724 (+ 79us) server_negotiation.cc:407] Connection header received
0809 09:33:17.734834 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:33:17.734836 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:33:17.734884 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:33:17.734957 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:33:17.735504 (+ 547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:33:17.736030 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:33:17.736913 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:33:17.737245 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:33:17.737370 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:33:17.740610 (+ 3240us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.125.130:54553: BlockingRecv error: recv got EOF from 127.20.125.130:54553 (error 108)
Metrics: {"server-negotiator.queue_time_us":202}
Aug 09 09:33:17 dist-test-slave-7kzw krb5kdc[799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.125.131@KRBTEST.COM for oryx/127.20.125.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 09:33:17 dist-test-slave-7kzw krb5kdc[799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.125.131@KRBTEST.COM for oryx/127.20.125.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 09:33:17.946923 2302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:33:17.940881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:40061 (local address 127.20.125.131:59985)
0809 09:33:17.941192 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0809 09:33:17.941206 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 09:33:17.941306 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:33:17.941677 (+ 371us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:33:17.941681 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:33:17.941898 (+ 217us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:33:17.942009 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:33:17.942016 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:33:17.942844 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:33:17.942847 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:33:17.943461 (+ 614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:33:17.943470 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:33:17.943920 (+ 450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:33:17.943924 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:33:17.943994 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:33:17.944512 (+ 518us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 09:33:17.944524 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:33:17.946792 (+ 2268us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.125.190:40061: Server oryx/127.20.125.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":229,"thread_start_us":94,"threads_started":1}
W20260809 09:33:17.947078 2297 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 09:33:17.940988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.131:59985 (local address 127.20.125.190:40061)
0809 09:33:17.941124 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0809 09:33:17.941127 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 09:33:17.941277 (+ 150us) server_negotiation.cc:407] Connection header received
0809 09:33:17.941386 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:33:17.941388 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:33:17.941436 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:33:17.941505 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:33:17.942132 (+ 627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:33:17.942737 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:33:17.943567 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:33:17.943792 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:33:17.943870 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:33:17.946947 (+ 3077us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.125.131:59985: BlockingRecv error: recv got EOF from 127.20.125.131:59985 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Aug 09 09:33:18 dist-test-slave-7kzw krb5kdc[799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.125.129@KRBTEST.COM for oryx/127.20.125.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 09:33:18 dist-test-slave-7kzw krb5kdc[799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.125.129@KRBTEST.COM for oryx/127.20.125.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 09:33:18.546026 2303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:33:18.540133 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:40061 (local address 127.20.125.129:53973)
0809 09:33:18.540427 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0809 09:33:18.540441 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 09:33:18.540530 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:33:18.540988 (+ 458us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:33:18.540990 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:33:18.541201 (+ 211us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:33:18.541308 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:33:18.541315 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:33:18.542161 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:33:18.542164 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:33:18.542788 (+ 624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:33:18.542797 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:33:18.543263 (+ 466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:33:18.543265 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:33:18.543329 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:33:18.543858 (+ 529us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 09:33:18.543870 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:33:18.545900 (+ 2030us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.125.190:40061: Server oryx/127.20.125.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":108,"threads_started":1}
W20260809 09:33:18.546171 2304 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 09:33:18.540360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.129:53973 (local address 127.20.125.190:40061)
0809 09:33:18.540673 (+ 313us) server_negotiation.cc:206] Beginning negotiation
0809 09:33:18.540677 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 09:33:18.540691 (+ 14us) server_negotiation.cc:407] Connection header received
0809 09:33:18.540727 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:33:18.540730 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:33:18.540776 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:33:18.540870 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:33:18.541437 (+ 567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:33:18.542046 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:33:18.542905 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:33:18.543154 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:33:18.543246 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:33:18.546035 (+ 2789us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.125.129:53973: BlockingRecv error: recv got EOF from 127.20.125.129:53973 (error 108)
Metrics: {"server-negotiator.queue_time_us":203,"thread_start_us":112,"threads_started":1}
Aug 09 09:33:18 dist-test-slave-7kzw krb5kdc[799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.125.130@KRBTEST.COM for oryx/127.20.125.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 09:33:18 dist-test-slave-7kzw krb5kdc[799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.125.130@KRBTEST.COM for oryx/127.20.125.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 09:33:18.747445 2305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:33:18.741460 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:40061 (local address 127.20.125.130:55893)
0809 09:33:18.741761 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0809 09:33:18.741774 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 09:33:18.741873 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:33:18.742106 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:33:18.742109 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:33:18.742308 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:33:18.742407 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:33:18.742413 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:33:18.743353 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:33:18.743356 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:33:18.743923 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:33:18.743931 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:33:18.744371 (+ 440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:33:18.744374 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:33:18.744437 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:33:18.744983 (+ 546us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 09:33:18.744997 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:33:18.747305 (+ 2308us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.125.190:40061: Server oryx/127.20.125.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":95,"threads_started":1}
W20260809 09:33:18.747779 2304 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 09:33:18.741583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.130:55893 (local address 127.20.125.190:40061)
0809 09:33:18.741737 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0809 09:33:18.741741 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 09:33:18.741839 (+ 98us) server_negotiation.cc:407] Connection header received
0809 09:33:18.741955 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:33:18.741958 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:33:18.742003 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:33:18.742063 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:33:18.742557 (+ 494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:33:18.743246 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:33:18.744032 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:33:18.744271 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:33:18.744351 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:33:18.747545 (+ 3194us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.125.130:55893: BlockingRecv error: recv got EOF from 127.20.125.130:55893 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Aug 09 09:33:18 dist-test-slave-7kzw krb5kdc[799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.125.131@KRBTEST.COM for oryx/127.20.125.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 09:33:18 dist-test-slave-7kzw krb5kdc[799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.125.131@KRBTEST.COM for oryx/127.20.125.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 09:33:18.953824 2308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:33:18.947878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:40061 (local address 127.20.125.131:60115)
0809 09:33:18.948187 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0809 09:33:18.948202 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 09:33:18.948314 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:33:18.948568 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:33:18.948571 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:33:18.948902 (+ 331us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:33:18.949012 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:33:18.949019 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:33:18.949980 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:33:18.949983 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:33:18.950700 (+ 717us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:33:18.950710 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:33:18.951163 (+ 453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:33:18.951166 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:33:18.951228 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:33:18.951739 (+ 511us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 09:33:18.951751 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:33:18.953661 (+ 1910us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.125.190:40061: Server oryx/127.20.125.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":115,"threads_started":1}
W20260809 09:33:18.953910 2304 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 09:33:18.947994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.131:60115 (local address 127.20.125.190:40061)
0809 09:33:18.948130 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0809 09:33:18.948134 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 09:33:18.948274 (+ 140us) server_negotiation.cc:407] Connection header received
0809 09:33:18.948396 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:33:18.948398 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:33:18.948449 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:33:18.948521 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:33:18.949169 (+ 648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:33:18.949830 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:33:18.950817 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:33:18.951055 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:33:18.951160 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:33:18.953808 (+ 2648us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.125.131:60115: BlockingRecv error: recv got EOF from 127.20.125.131:60115 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Aug 09 09:33:19 dist-test-slave-7kzw krb5kdc[799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.125.129@KRBTEST.COM for oryx/127.20.125.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 09:33:19 dist-test-slave-7kzw krb5kdc[799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.125.129@KRBTEST.COM for oryx/127.20.125.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 09:33:19.552094 2309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:33:19.546884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:40061 (local address 127.20.125.129:57643)
0809 09:33:19.547187 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0809 09:33:19.547203 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 09:33:19.547306 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:33:19.547539 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:33:19.547541 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:33:19.547742 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:33:19.547840 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:33:19.547846 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:33:19.548569 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:33:19.548572 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:33:19.549114 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:33:19.549121 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:33:19.549528 (+ 407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:33:19.549531 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:33:19.549592 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:33:19.550075 (+ 483us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 09:33:19.550087 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:33:19.551978 (+ 1891us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.125.190:40061: Server oryx/127.20.125.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":77,"threads_started":1}
W20260809 09:33:19.552330 2310 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 09:33:19.547018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.129:57643 (local address 127.20.125.190:40061)
0809 09:33:19.547250 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0809 09:33:19.547254 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 09:33:19.547266 (+ 12us) server_negotiation.cc:407] Connection header received
0809 09:33:19.547387 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:33:19.547389 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:33:19.547432 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:33:19.547500 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:33:19.547947 (+ 447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:33:19.548474 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:33:19.549215 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:33:19.549430 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:33:19.549504 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:33:19.552149 (+ 2645us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.125.129:57643: BlockingRecv error: recv got EOF from 127.20.125.129:57643 (error 108)
Metrics: {"server-negotiator.queue_time_us":160,"thread_start_us":80,"threads_started":1}
Aug 09 09:33:19 dist-test-slave-7kzw krb5kdc[799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.125.130@KRBTEST.COM for oryx/127.20.125.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 09:33:19 dist-test-slave-7kzw krb5kdc[799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.125.130@KRBTEST.COM for oryx/127.20.125.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 09:33:19.754727 2312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:33:19.748383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:40061 (local address 127.20.125.130:60769)
0809 09:33:19.748730 (+ 347us) negotiation.cc:107] Waiting for socket to connect
0809 09:33:19.748747 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 09:33:19.748868 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:33:19.749194 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:33:19.749197 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:33:19.749424 (+ 227us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:33:19.749521 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:33:19.749527 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:33:19.750326 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:33:19.750331 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:33:19.750930 (+ 599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:33:19.750938 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:33:19.751404 (+ 466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:33:19.751408 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:33:19.751496 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:33:19.752120 (+ 624us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 09:33:19.752150 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:33:19.754550 (+ 2400us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.125.190:40061: Server oryx/127.20.125.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":263,"thread_start_us":110,"threads_started":1}
W20260809 09:33:19.754823 2310 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 09:33:19.748495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.130:60769 (local address 127.20.125.190:40061)
0809 09:33:19.748712 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0809 09:33:19.748717 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 09:33:19.748808 (+ 91us) server_negotiation.cc:407] Connection header received
0809 09:33:19.748986 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:33:19.748990 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:33:19.749049 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:33:19.749151 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:33:19.749647 (+ 496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:33:19.750211 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:33:19.751053 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:33:19.751289 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:33:19.751401 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:33:19.754706 (+ 3305us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.125.130:60769: BlockingRecv error: recv got EOF from 127.20.125.130:60769 (error 108)
Metrics: {"server-negotiator.queue_time_us":93}
Aug 09 09:33:19 dist-test-slave-7kzw krb5kdc[799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.125.131@KRBTEST.COM for oryx/127.20.125.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 09:33:19 dist-test-slave-7kzw krb5kdc[799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.125.131@KRBTEST.COM for oryx/127.20.125.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 09:33:19.960656 2313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:33:19.954695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:40061 (local address 127.20.125.131:37105)
0809 09:33:19.955047 (+ 352us) negotiation.cc:107] Waiting for socket to connect
0809 09:33:19.955065 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 09:33:19.955220 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:33:19.955507 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:33:19.955510 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:33:19.955738 (+ 228us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:33:19.955831 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:33:19.955836 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:33:19.956609 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:33:19.956614 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:33:19.957319 (+ 705us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:33:19.957329 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:33:19.957804 (+ 475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:33:19.957807 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:33:19.957896 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:33:19.958457 (+ 561us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 09:33:19.958519 (+ 62us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:33:19.960483 (+ 1964us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.125.190:40061: Server oryx/127.20.125.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":275,"thread_start_us":114,"threads_started":1}
W20260809 09:33:19.960768 2310 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 09:33:19.954829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.131:37105 (local address 127.20.125.190:40061)
0809 09:33:19.955019 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0809 09:33:19.955032 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 09:33:19.955156 (+ 124us) server_negotiation.cc:407] Connection header received
0809 09:33:19.955323 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:33:19.955327 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:33:19.955396 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:33:19.955485 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:33:19.955956 (+ 471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:33:19.956497 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:33:19.957447 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:33:19.957685 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:33:19.957800 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:33:19.960614 (+ 2814us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.125.131:37105: BlockingRecv error: recv got EOF from 127.20.125.131:37105 (error 108)
Metrics: {"server-negotiator.queue_time_us":88}
Aug 09 09:33:20 dist-test-slave-7kzw krb5kdc[799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.125.129@KRBTEST.COM for oryx/127.20.125.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 09:33:20 dist-test-slave-7kzw krb5kdc[799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.125.129@KRBTEST.COM for oryx/127.20.125.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 09:33:20.558660 2316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:33:20.552981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:40061 (local address 127.20.125.129:60053)
0809 09:33:20.553311 (+ 330us) negotiation.cc:107] Waiting for socket to connect
0809 09:33:20.553329 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 09:33:20.553435 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:33:20.553669 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:33:20.553671 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:33:20.553866 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:33:20.553962 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:33:20.553967 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:33:20.554740 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:33:20.554743 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:33:20.555312 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:33:20.555321 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:33:20.555750 (+ 429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:33:20.555754 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:33:20.555821 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:33:20.556318 (+ 497us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 09:33:20.556333 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:33:20.558460 (+ 2127us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.125.190:40061: Server oryx/127.20.125.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":96,"threads_started":1}
W20260809 09:33:20.558768 2317 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 09:33:20.553187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.129:60053 (local address 127.20.125.190:40061)
0809 09:33:20.553450 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0809 09:33:20.553454 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 09:33:20.553468 (+ 14us) server_negotiation.cc:407] Connection header received
0809 09:33:20.553504 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:33:20.553506 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:33:20.553549 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:33:20.553624 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:33:20.554070 (+ 446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:33:20.554641 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:33:20.555416 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:33:20.555639 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:33:20.555726 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:33:20.558629 (+ 2903us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.125.129:60053: BlockingRecv error: recv got EOF from 127.20.125.129:60053 (error 108)
Metrics: {"server-negotiator.queue_time_us":188,"thread_start_us":79,"threads_started":1}
Aug 09 09:33:20 dist-test-slave-7kzw krb5kdc[799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.125.130@KRBTEST.COM for oryx/127.20.125.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 09:33:20 dist-test-slave-7kzw krb5kdc[799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.125.130@KRBTEST.COM for oryx/127.20.125.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 09:33:20.761931 2318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:33:20.755677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:40061 (local address 127.20.125.130:44419)
0809 09:33:20.756007 (+ 330us) negotiation.cc:107] Waiting for socket to connect
0809 09:33:20.756027 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 09:33:20.756158 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:33:20.756458 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:33:20.756461 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:33:20.756693 (+ 232us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:33:20.756795 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:33:20.756801 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:33:20.757680 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:33:20.757685 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:33:20.758387 (+ 702us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:33:20.758398 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:33:20.759101 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:33:20.759104 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:33:20.759197 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:33:20.759816 (+ 619us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 09:33:20.759829 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:33:20.761784 (+ 1955us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.125.190:40061: Server oryx/127.20.125.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":116,"threads_started":1}
W20260809 09:33:20.762019 2317 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 09:33:20.755769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.130:44419 (local address 127.20.125.190:40061)
0809 09:33:20.755958 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0809 09:33:20.755963 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 09:33:20.756104 (+ 141us) server_negotiation.cc:407] Connection header received
0809 09:33:20.756296 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:33:20.756298 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:33:20.756345 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:33:20.756432 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:33:20.756922 (+ 490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:33:20.757564 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:33:20.758624 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:33:20.758937 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:33:20.759067 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:33:20.761915 (+ 2848us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.125.130:44419: BlockingRecv error: recv got EOF from 127.20.125.130:44419 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
Aug 09 09:33:20 dist-test-slave-7kzw krb5kdc[799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.125.131@KRBTEST.COM for oryx/127.20.125.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 09:33:20 dist-test-slave-7kzw krb5kdc[799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.125.131@KRBTEST.COM for oryx/127.20.125.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 09:33:20.967455 2319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:33:20.961500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:40061 (local address 127.20.125.131:57299)
0809 09:33:20.961825 (+ 325us) negotiation.cc:107] Waiting for socket to connect
0809 09:33:20.961842 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 09:33:20.961954 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:33:20.962264 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:33:20.962267 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:33:20.962520 (+ 253us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:33:20.962615 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:33:20.962621 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:33:20.963409 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:33:20.963414 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:33:20.963996 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:33:20.964004 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:33:20.964493 (+ 489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:33:20.964497 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:33:20.964588 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:33:20.965094 (+ 506us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 09:33:20.965106 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:33:20.967264 (+ 2158us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.125.190:40061: Server oryx/127.20.125.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":101,"threads_started":1}
W20260809 09:33:20.967522 2317 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 09:33:20.961663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.131:57299 (local address 127.20.125.190:40061)
0809 09:33:20.961821 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0809 09:33:20.961825 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 09:33:20.961903 (+ 78us) server_negotiation.cc:407] Connection header received
0809 09:33:20.962091 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:33:20.962094 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:33:20.962142 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:33:20.962238 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:33:20.962774 (+ 536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:33:20.963293 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:33:20.964120 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:33:20.964371 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:33:20.964488 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:33:20.967412 (+ 2924us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.125.131:57299: BlockingRecv error: recv got EOF from 127.20.125.131:57299 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Aug 09 09:33:21 dist-test-slave-7kzw krb5kdc[799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.125.129@KRBTEST.COM for oryx/127.20.125.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 09:33:21 dist-test-slave-7kzw krb5kdc[799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.125.129@KRBTEST.COM for oryx/127.20.125.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 09:33:21.565693 2321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:33:21.559590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:40061 (local address 127.20.125.129:58269)
0809 09:33:21.559972 (+ 382us) negotiation.cc:107] Waiting for socket to connect
0809 09:33:21.559991 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 09:33:21.560137 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:33:21.560560 (+ 423us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:33:21.560563 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:33:21.560843 (+ 280us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:33:21.560979 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:33:21.560986 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:33:21.561776 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:33:21.561781 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:33:21.562357 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:33:21.562365 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:33:21.562870 (+ 505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:33:21.562874 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:33:21.562962 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:33:21.563536 (+ 574us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 09:33:21.563557 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:33:21.565483 (+ 1926us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.125.190:40061: Server oryx/127.20.125.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":274,"thread_start_us":149,"threads_started":1}
W20260809 09:33:21.565733 2322 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 09:33:21.559810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.129:58269 (local address 127.20.125.190:40061)
0809 09:33:21.560139 (+ 329us) server_negotiation.cc:206] Beginning negotiation
0809 09:33:21.560144 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 09:33:21.560158 (+ 14us) server_negotiation.cc:407] Connection header received
0809 09:33:21.560282 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:33:21.560288 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:33:21.560385 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:33:21.560538 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:33:21.561106 (+ 568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:33:21.561659 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:33:21.562505 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:33:21.562750 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:33:21.562866 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:33:21.565609 (+ 2743us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.125.129:58269: BlockingRecv error: recv got EOF from 127.20.125.129:58269 (error 108)
Metrics: {"server-negotiator.queue_time_us":227,"thread_start_us":109,"threads_started":1}
Aug 09 09:33:21 dist-test-slave-7kzw krb5kdc[799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.125.130@KRBTEST.COM for oryx/127.20.125.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 09:33:21 dist-test-slave-7kzw krb5kdc[799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.125.130@KRBTEST.COM for oryx/127.20.125.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 09:33:21.769106 2324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:33:21.763278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:40061 (local address 127.20.125.130:50943)
0809 09:33:21.763623 (+ 345us) negotiation.cc:107] Waiting for socket to connect
0809 09:33:21.763639 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 09:33:21.763795 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:33:21.764083 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:33:21.764086 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:33:21.764323 (+ 237us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:33:21.764421 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:33:21.764427 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:33:21.765218 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:33:21.765223 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:33:21.765786 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:33:21.765794 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:33:21.766275 (+ 481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:33:21.766278 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:33:21.766359 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:33:21.766987 (+ 628us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 09:33:21.766999 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:33:21.768924 (+ 1925us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.125.190:40061: Server oryx/127.20.125.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":263,"thread_start_us":109,"threads_started":1}
W20260809 09:33:21.769160 2322 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 09:33:21.763364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.130:50943 (local address 127.20.125.190:40061)
0809 09:33:21.763549 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0809 09:33:21.763554 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 09:33:21.763731 (+ 177us) server_negotiation.cc:407] Connection header received
0809 09:33:21.763908 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:33:21.763911 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:33:21.763966 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:33:21.764058 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:33:21.764551 (+ 493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:33:21.765096 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:33:21.765912 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:33:21.766152 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:33:21.766269 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:33:21.769053 (+ 2784us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.125.130:50943: BlockingRecv error: recv got EOF from 127.20.125.130:50943 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
Aug 09 09:33:21 dist-test-slave-7kzw krb5kdc[799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.125.131@KRBTEST.COM for oryx/127.20.125.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 09:33:21 dist-test-slave-7kzw krb5kdc[799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.125.131@KRBTEST.COM for oryx/127.20.125.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 09:33:21.975031 2325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:33:21.968488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:40061 (local address 127.20.125.131:45949)
0809 09:33:21.968838 (+ 350us) negotiation.cc:107] Waiting for socket to connect
0809 09:33:21.968863 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0809 09:33:21.969073 (+ 210us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:33:21.969561 (+ 488us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:33:21.969564 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:33:21.969819 (+ 255us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:33:21.969926 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:33:21.969933 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:33:21.970863 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:33:21.970870 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:33:21.971499 (+ 629us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:33:21.971507 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:33:21.972031 (+ 524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:33:21.972035 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:33:21.972135 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:33:21.972748 (+ 613us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 09:33:21.972762 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:33:21.974836 (+ 2074us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.125.190:40061: Server oryx/127.20.125.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":250,"thread_start_us":140,"threads_started":1}
W20260809 09:33:21.975095 2322 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 09:33:21.968615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.131:45949 (local address 127.20.125.190:40061)
0809 09:33:21.968808 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0809 09:33:21.968813 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 09:33:21.969254 (+ 441us) server_negotiation.cc:407] Connection header received
0809 09:33:21.969347 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:33:21.969351 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:33:21.969419 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:33:21.969523 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:33:21.970071 (+ 548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:33:21.970736 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:33:21.971635 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:33:21.971894 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:33:21.972015 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:33:21.974974 (+ 2959us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.125.131:45949: BlockingRecv error: recv got EOF from 127.20.125.131:45949 (error 108)
Metrics: {"server-negotiator.queue_time_us":100}
Aug 09 09:33:22 dist-test-slave-7kzw krb5kdc[799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.125.129@KRBTEST.COM for oryx/127.20.125.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 09:33:22 dist-test-slave-7kzw krb5kdc[799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.125.129@KRBTEST.COM for oryx/127.20.125.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 09:33:22.575332 2327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:33:22.566569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:40061 (local address 127.20.125.129:36117)
0809 09:33:22.566930 (+ 361us) negotiation.cc:107] Waiting for socket to connect
0809 09:33:22.566948 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 09:33:22.567055 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:33:22.567338 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:33:22.567340 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:33:22.567537 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:33:22.567669 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:33:22.567675 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:33:22.568429 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:33:22.568431 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:33:22.569044 (+ 613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:33:22.569054 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:33:22.569512 (+ 458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:33:22.569515 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:33:22.569581 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:33:22.573200 (+ 3619us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 09:33:22.573213 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:33:22.575208 (+ 1995us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.125.190:40061: Server oryx/127.20.125.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":275,"thread_start_us":137,"threads_started":1}
W20260809 09:33:22.575516 2328 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 09:33:22.566655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.129:36117 (local address 127.20.125.190:40061)
0809 09:33:22.566931 (+ 276us) server_negotiation.cc:206] Beginning negotiation
0809 09:33:22.566935 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 09:33:22.567104 (+ 169us) server_negotiation.cc:407] Connection header received
0809 09:33:22.567153 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:33:22.567155 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:33:22.567207 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:33:22.567304 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:33:22.567787 (+ 483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:33:22.568324 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:33:22.569173 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:33:22.569402 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:33:22.569486 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:33:22.575367 (+ 5881us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.125.129:36117: BlockingRecv error: recv got EOF from 127.20.125.129:36117 (error 108)
Metrics: {"server-negotiator.queue_time_us":186,"thread_start_us":83,"threads_started":1}
Aug 09 09:33:22 dist-test-slave-7kzw krb5kdc[799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.125.130@KRBTEST.COM for oryx/127.20.125.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 09:33:22 dist-test-slave-7kzw krb5kdc[799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.125.130@KRBTEST.COM for oryx/127.20.125.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 09:33:22.776410 2329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:33:22.770011 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:40061 (local address 127.20.125.130:34595)
0809 09:33:22.770338 (+ 327us) negotiation.cc:107] Waiting for socket to connect
0809 09:33:22.770356 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 09:33:22.770505 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:33:22.770941 (+ 436us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:33:22.770949 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:33:22.771283 (+ 334us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:33:22.771437 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:33:22.771446 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:33:22.772444 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:33:22.772447 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:33:22.773089 (+ 642us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:33:22.773098 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:33:22.773637 (+ 539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:33:22.773641 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:33:22.773717 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:33:22.774294 (+ 577us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 09:33:22.774305 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:33:22.776290 (+ 1985us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.125.190:40061: Server oryx/127.20.125.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":246,"thread_start_us":161,"threads_started":1}
W20260809 09:33:22.776743 2328 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 09:33:22.770169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.130:34595 (local address 127.20.125.190:40061)
0809 09:33:22.770328 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0809 09:33:22.770334 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 09:33:22.770447 (+ 113us) server_negotiation.cc:407] Connection header received
0809 09:33:22.770621 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:33:22.770624 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:33:22.770706 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:33:22.770821 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:33:22.771575 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:33:22.772326 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:33:22.773233 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:33:22.773491 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:33:22.773598 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:33:22.776452 (+ 2854us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.125.130:34595: BlockingRecv error: recv got EOF from 127.20.125.130:34595 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 09 09:33:22 dist-test-slave-7kzw krb5kdc[799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.125.131@KRBTEST.COM for oryx/127.20.125.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 09:33:22 dist-test-slave-7kzw krb5kdc[799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.125.131@KRBTEST.COM for oryx/127.20.125.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 09:33:22.981943 2330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:33:22.976005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:40061 (local address 127.20.125.131:32791)
0809 09:33:22.976304 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0809 09:33:22.976319 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 09:33:22.976426 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:33:22.976692 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:33:22.976694 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:33:22.976903 (+ 209us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:33:22.977007 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:33:22.977012 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:33:22.977817 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:33:22.977820 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:33:22.978431 (+ 611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:33:22.978440 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:33:22.978995 (+ 555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:33:22.978998 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:33:22.979069 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:33:22.979669 (+ 600us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 09:33:22.979682 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:33:22.981809 (+ 2127us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.125.190:40061: Server oryx/127.20.125.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":104,"threads_started":1}
W20260809 09:33:22.982124 2328 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 09:33:22.976063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.131:32791 (local address 127.20.125.190:40061)
0809 09:33:22.976245 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0809 09:33:22.976249 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 09:33:22.976397 (+ 148us) server_negotiation.cc:407] Connection header received
0809 09:33:22.976517 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:33:22.976519 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:33:22.976570 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:33:22.976645 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:33:22.977116 (+ 471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:33:22.977709 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:33:22.978602 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:33:22.978877 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:33:22.978971 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:33:22.981977 (+ 3006us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.125.131:32791: BlockingRecv error: recv got EOF from 127.20.125.131:32791 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
Aug 09 09:33:23 dist-test-slave-7kzw krb5kdc[799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.125.129@KRBTEST.COM for oryx/127.20.125.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 09:33:23 dist-test-slave-7kzw krb5kdc[799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.125.129@KRBTEST.COM for oryx/127.20.125.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 09:33:23.582103 2331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:33:23.576196 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:40061 (local address 127.20.125.129:39161)
0809 09:33:23.576582 (+ 386us) negotiation.cc:107] Waiting for socket to connect
0809 09:33:23.576597 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 09:33:23.576705 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:33:23.576959 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:33:23.576961 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:33:23.577161 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:33:23.577259 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:33:23.577264 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:33:23.578197 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:33:23.578202 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:33:23.578809 (+ 607us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:33:23.578818 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:33:23.579321 (+ 503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:33:23.579326 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:33:23.579411 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:33:23.580078 (+ 667us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 09:33:23.580094 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:33:23.581957 (+ 1863us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.125.190:40061: Server oryx/127.20.125.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":91,"threads_started":1}
W20260809 09:33:23.582305 2333 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 09:33:23.576317 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.129:39161 (local address 127.20.125.190:40061)
0809 09:33:23.576560 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0809 09:33:23.576563 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 09:33:23.576701 (+ 138us) server_negotiation.cc:407] Connection header received
0809 09:33:23.576785 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:33:23.576787 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:33:23.576834 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:33:23.576912 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:33:23.577437 (+ 525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:33:23.578068 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:33:23.578924 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:33:23.579178 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:33:23.579268 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:33:23.582115 (+ 2847us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.125.129:39161: BlockingRecv error: recv got EOF from 127.20.125.129:39161 (error 108)
Metrics: {"server-negotiator.queue_time_us":163,"thread_start_us":64,"threads_started":1}
Aug 09 09:33:23 dist-test-slave-7kzw krb5kdc[799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.125.130@KRBTEST.COM for oryx/127.20.125.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 09:33:23 dist-test-slave-7kzw krb5kdc[799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.125.130@KRBTEST.COM for oryx/127.20.125.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 09:33:23.783138 2334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:33:23.777263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:40061 (local address 127.20.125.130:56285)
0809 09:33:23.777557 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0809 09:33:23.777571 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 09:33:23.777663 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:33:23.777931 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:33:23.777933 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:33:23.778127 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:33:23.778243 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:33:23.778249 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:33:23.779167 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:33:23.779169 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:33:23.779727 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:33:23.779736 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:33:23.780247 (+ 511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:33:23.780250 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:33:23.780323 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:33:23.780925 (+ 602us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 09:33:23.780937 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:33:23.783018 (+ 2081us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.125.190:40061: Server oryx/127.20.125.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":108,"threads_started":1}
W20260809 09:33:23.783316 2333 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 09:33:23.777408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.130:56285 (local address 127.20.125.190:40061)
0809 09:33:23.777586 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0809 09:33:23.777592 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 09:33:23.777613 (+ 21us) server_negotiation.cc:407] Connection header received
0809 09:33:23.777761 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:33:23.777763 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:33:23.777814 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:33:23.777893 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:33:23.778366 (+ 473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:33:23.779042 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:33:23.779860 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:33:23.780101 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:33:23.780210 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:33:23.783184 (+ 2974us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.125.130:56285: BlockingRecv error: recv got EOF from 127.20.125.130:56285 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
Aug 09 09:33:23 dist-test-slave-7kzw krb5kdc[799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.125.131@KRBTEST.COM for oryx/127.20.125.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 09:33:23 dist-test-slave-7kzw krb5kdc[799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.125.131@KRBTEST.COM for oryx/127.20.125.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 09:33:23.989334 2337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:33:23.983015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:40061 (local address 127.20.125.131:48219)
0809 09:33:23.983355 (+ 340us) negotiation.cc:107] Waiting for socket to connect
0809 09:33:23.983370 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 09:33:23.983513 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:33:23.983817 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:33:23.983820 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:33:23.984045 (+ 225us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:33:23.984141 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:33:23.984146 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:33:23.984901 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:33:23.984906 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:33:23.985688 (+ 782us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:33:23.985703 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:33:23.986373 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:33:23.986377 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:33:23.986500 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:33:23.987099 (+ 599us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 09:33:23.987111 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:33:23.989203 (+ 2092us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.125.190:40061: Server oryx/127.20.125.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":105,"threads_started":1}
W20260809 09:33:23.989464 2333 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 09:33:23.983086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.131:48219 (local address 127.20.125.190:40061)
0809 09:33:23.983312 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0809 09:33:23.983317 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 09:33:23.983460 (+ 143us) server_negotiation.cc:407] Connection header received
0809 09:33:23.983621 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:33:23.983624 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:33:23.983687 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:33:23.983790 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:33:23.984260 (+ 470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:33:23.984790 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:33:23.985848 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:33:23.986241 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:33:23.986365 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:33:23.989342 (+ 2977us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.125.131:48219: BlockingRecv error: recv got EOF from 127.20.125.131:48219 (error 108)
Metrics: {"server-negotiator.queue_time_us":125}
Aug 09 09:33:24 dist-test-slave-7kzw krb5kdc[799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.125.129@KRBTEST.COM for oryx/127.20.125.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 09:33:24 dist-test-slave-7kzw krb5kdc[799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.125.129@KRBTEST.COM for oryx/127.20.125.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 09:33:24.588858 2340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:33:24.583010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:40061 (local address 127.20.125.129:35117)
0809 09:33:24.583367 (+ 357us) negotiation.cc:107] Waiting for socket to connect
0809 09:33:24.583388 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 09:33:24.583488 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:33:24.583722 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:33:24.583724 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:33:24.583919 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:33:24.584017 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:33:24.584031 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:33:24.584870 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:33:24.584873 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:33:24.585458 (+ 585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:33:24.585467 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:33:24.585989 (+ 522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:33:24.585993 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:33:24.586060 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:33:24.586667 (+ 607us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 09:33:24.586681 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:33:24.588645 (+ 1964us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.125.190:40061: Server oryx/127.20.125.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":265,"thread_start_us":102,"threads_started":1}
W20260809 09:33:24.588985 2342 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 09:33:24.583158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.129:35117 (local address 127.20.125.190:40061)
0809 09:33:24.583517 (+ 359us) server_negotiation.cc:206] Beginning negotiation
0809 09:33:24.583521 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 09:33:24.583536 (+ 15us) server_negotiation.cc:407] Connection header received
0809 09:33:24.583575 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:33:24.583577 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:33:24.583622 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:33:24.583710 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:33:24.584149 (+ 439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:33:24.584743 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:33:24.585618 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:33:24.585870 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:33:24.585962 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:33:24.588824 (+ 2862us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.125.129:35117: BlockingRecv error: recv got EOF from 127.20.125.129:35117 (error 108)
Metrics: {"server-negotiator.queue_time_us":277,"thread_start_us":78,"threads_started":1}
Aug 09 09:33:24 dist-test-slave-7kzw krb5kdc[799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.125.130@KRBTEST.COM for oryx/127.20.125.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 09:33:24 dist-test-slave-7kzw krb5kdc[799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.125.130@KRBTEST.COM for oryx/127.20.125.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 09:33:24.789857 2346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:33:24.783984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:40061 (local address 127.20.125.130:46197)
0809 09:33:24.784276 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0809 09:33:24.784290 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 09:33:24.784392 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:33:24.784627 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:33:24.784629 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:33:24.784861 (+ 232us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:33:24.784960 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:33:24.784966 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:33:24.785742 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:33:24.785744 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:33:24.786301 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:33:24.786309 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:33:24.786863 (+ 554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:33:24.786865 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:33:24.786933 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:33:24.787460 (+ 527us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 09:33:24.787472 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:33:24.789650 (+ 2178us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.125.190:40061: Server oryx/127.20.125.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":111,"threads_started":1}
W20260809 09:33:24.789969 2342 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 09:33:24.784082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.130:46197 (local address 127.20.125.190:40061)
0809 09:33:24.784226 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0809 09:33:24.784230 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 09:33:24.784385 (+ 155us) server_negotiation.cc:407] Connection header received
0809 09:33:24.784461 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:33:24.784463 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:33:24.784512 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:33:24.784583 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:33:24.785070 (+ 487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:33:24.785623 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:33:24.786442 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:33:24.786738 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:33:24.786831 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:33:24.789844 (+ 3013us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.125.130:46197: BlockingRecv error: recv got EOF from 127.20.125.130:46197 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Aug 09 09:33:24 dist-test-slave-7kzw krb5kdc[799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.125.131@KRBTEST.COM for oryx/127.20.125.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 09:33:24 dist-test-slave-7kzw krb5kdc[799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.125.131@KRBTEST.COM for oryx/127.20.125.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 09:33:24.997087 2348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:33:24.990259 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:40061 (local address 127.20.125.131:40053)
0809 09:33:24.990695 (+ 436us) negotiation.cc:107] Waiting for socket to connect
0809 09:33:24.990714 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 09:33:24.990846 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:33:24.991162 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:33:24.991164 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:33:24.991363 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:33:24.991466 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:33:24.991473 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:33:24.992719 (+ 1246us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:33:24.992722 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:33:24.993286 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:33:24.993295 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:33:24.994036 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:33:24.994039 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:33:24.994116 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:33:24.994850 (+ 734us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 09:33:24.994867 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:33:24.996958 (+ 2091us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.125.190:40061: Server oryx/127.20.125.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":347,"thread_start_us":94,"threads_started":1}
W20260809 09:33:24.997383 2342 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 09:33:24.990372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.131:40053 (local address 127.20.125.190:40061)
0809 09:33:24.990569 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0809 09:33:24.990576 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 09:33:24.990810 (+ 234us) server_negotiation.cc:407] Connection header received
0809 09:33:24.990940 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:33:24.990945 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:33:24.991020 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:33:24.991119 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:33:24.991597 (+ 478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:33:24.992568 (+ 971us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:33:24.993413 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:33:24.993881 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:33:24.994014 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:33:24.997198 (+ 3184us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.125.131:40053: BlockingRecv error: recv got EOF from 127.20.125.131:40053 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
Aug 09 09:33:25 dist-test-slave-7kzw krb5kdc[799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.125.129@KRBTEST.COM for oryx/127.20.125.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 09:33:25 dist-test-slave-7kzw krb5kdc[799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.125.129@KRBTEST.COM for oryx/127.20.125.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 09:33:25.596127 2350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:33:25.589969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:40061 (local address 127.20.125.129:41101)
0809 09:33:25.590327 (+ 358us) negotiation.cc:107] Waiting for socket to connect
0809 09:33:25.590347 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 09:33:25.590508 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:33:25.590929 (+ 421us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:33:25.590933 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:33:25.591192 (+ 259us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:33:25.591319 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:33:25.591326 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:33:25.592367 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:33:25.592369 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:33:25.592951 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:33:25.592959 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:33:25.593380 (+ 421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:33:25.593383 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:33:25.593445 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:33:25.593952 (+ 507us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 09:33:25.593964 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:33:25.595977 (+ 2013us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.125.190:40061: Server oryx/127.20.125.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":245,"thread_start_us":87,"threads_started":1}
W20260809 09:33:25.596427 2354 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 09:33:25.590152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.129:41101 (local address 127.20.125.190:40061)
0809 09:33:25.590521 (+ 369us) server_negotiation.cc:206] Beginning negotiation
0809 09:33:25.590526 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 09:33:25.590547 (+ 21us) server_negotiation.cc:407] Connection header received
0809 09:33:25.590680 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:33:25.590682 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:33:25.590755 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:33:25.590861 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:33:25.591481 (+ 620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:33:25.592258 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:33:25.593052 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:33:25.593278 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:33:25.593357 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:33:25.596150 (+ 2793us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.125.129:41101: BlockingRecv error: recv got EOF from 127.20.125.129:41101 (error 108)
Metrics: {"server-negotiator.queue_time_us":238,"thread_start_us":107,"threads_started":1}
Aug 09 09:33:25 dist-test-slave-7kzw krb5kdc[799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.125.130@KRBTEST.COM for oryx/127.20.125.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 09:33:25 dist-test-slave-7kzw krb5kdc[799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.125.130@KRBTEST.COM for oryx/127.20.125.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 09:33:25.797122 2355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:33:25.791081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:40061 (local address 127.20.125.130:42617)
0809 09:33:25.791412 (+ 331us) negotiation.cc:107] Waiting for socket to connect
0809 09:33:25.791434 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 09:33:25.791537 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:33:25.791811 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:33:25.791814 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:33:25.792015 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:33:25.792133 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:33:25.792139 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:33:25.792974 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:33:25.792977 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:33:25.793543 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:33:25.793553 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:33:25.794046 (+ 493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:33:25.794050 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:33:25.794136 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:33:25.794872 (+ 736us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 09:33:25.794885 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:33:25.796980 (+ 2095us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.125.190:40061: Server oryx/127.20.125.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":250,"thread_start_us":120,"threads_started":1}
W20260809 09:33:25.797281 2354 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 09:33:25.791114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.130:42617 (local address 127.20.125.190:40061)
0809 09:33:25.791292 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0809 09:33:25.791298 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 09:33:25.791526 (+ 228us) server_negotiation.cc:407] Connection header received
0809 09:33:25.791615 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:33:25.791618 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:33:25.791674 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:33:25.791759 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:33:25.792280 (+ 521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:33:25.792834 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:33:25.793690 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:33:25.793920 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:33:25.794007 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:33:25.797140 (+ 3133us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.125.130:42617: BlockingRecv error: recv got EOF from 127.20.125.130:42617 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
I20260809 09:33:25.883090 20982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu with pid 1672
I20260809 09:33:25.899303 20982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu with pid 1892
I20260809 09:33:25.910310 20982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu with pid 2078
I20260809 09:33:25.923791 20982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu with pid 1574
2026-08-09T09:33:25Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (18443 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-09T09:33:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T09:33:25Z Disabled control of system clock
I20260809 09:33:25.963467 20982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
/tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786267941944493-20982-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786267941944493-20982-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786267941944493-20982-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786267941944493-20982-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.125.190:36357
--webserver_interface=127.20.125.190
--webserver_port=0
--builtin_ntp_servers=127.20.125.148:44839
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.125.190:36357
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260809 09:33:26.081888 2359 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 09:33:26.082171 2359 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 09:33:26.082206 2359 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 09:33:26.086196 2359 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 09:33:26.086261 2359 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 09:33:26.086277 2359 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 09:33:26.086297 2359 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 09:33:26.086311 2359 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 09:33:26.091260 2359 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.125.148:44839
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786267941944493-20982-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786267941944493-20982-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.125.190:36357
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.125.190:36357
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786267941944493-20982-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.125.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2359
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786267941944493-20982-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 08:38:55 UTC on ccycloud.quasar-njjleu.root.comops.site
build id 4575935
I20260809 09:33:26.092413 2359 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 09:33:26.093921 2359 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 09:33:26.103451 2370 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:33:26.103493 2368 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:33:26.103564 2359 server_base.cc:1034] running on GCE node
W20260809 09:33:26.103507 2372 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:33:26.104264 2359 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 09:33:26.105336 2359 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 09:33:26.106544 2359 hybrid_clock.cc:648] HybridClock initialized: now 1786268006106513 us; error 43 us; skew 500 ppm
I20260809 09:33:26.108511 2359 webserver.cc:460] Webserver started at http://127.20.125.190:45277/ using document root <none> and password file <none>
I20260809 09:33:26.109077 2359 fs_manager.cc:359] Metadata directory not provided
I20260809 09:33:26.109141 2359 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 09:33:26.109313 2359 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 09:33:26.111490 2359 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786267941944493-20982-0/minicluster-data/master-0/data/instance:
uuid: "9e13e5aa055847c892299e4c0d1c45d8"
format_stamp: "Formatted at 2026-08-09 09:33:26 on dist-test-slave-7kzw"
server_key: "de43c30a035f938e21fa0563fdd07812"
server_key_iv: "2e3ec9b9e1e3d1f00158aed8734d457d"
server_key_version: "encryptionkey@0"
I20260809 09:33:26.112013 2359 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786267941944493-20982-0/minicluster-data/master-0/wal/instance:
uuid: "9e13e5aa055847c892299e4c0d1c45d8"
format_stamp: "Formatted at 2026-08-09 09:33:26 on dist-test-slave-7kzw"
server_key: "de43c30a035f938e21fa0563fdd07812"
server_key_iv: "2e3ec9b9e1e3d1f00158aed8734d457d"
server_key_version: "encryptionkey@0"
I20260809 09:33:26.115588 2359 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.004s
I20260809 09:33:26.117956 2378 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 09:33:26.119264 2359 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260809 09:33:26.119366 2359 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786267941944493-20982-0/minicluster-data/master-0/data,/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786267941944493-20982-0/minicluster-data/master-0/wal
uuid: "9e13e5aa055847c892299e4c0d1c45d8"
format_stamp: "Formatted at 2026-08-09 09:33:26 on dist-test-slave-7kzw"
server_key: "de43c30a035f938e21fa0563fdd07812"
server_key_iv: "2e3ec9b9e1e3d1f00158aed8734d457d"
server_key_version: "encryptionkey@0"
I20260809 09:33:26.119453 2359 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786267941944493-20982-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786267941944493-20982-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786267941944493-20982-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 09:33:26.179757 2359 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 09:33:26.180516 2359 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 09:33:26.180755 2359 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 09:33:26.188123 2359 rpc_server.cc:307] RPC server started. Bound to: 127.20.125.190:36357
I20260809 09:33:26.188138 2452 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.125.190:36357 every 8 connection(s)
I20260809 09:33:26.189082 2359 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786267941944493-20982-0/minicluster-data/master-0/data/info.pb
I20260809 09:33:26.190965 20982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu as pid 2359
I20260809 09:33:26.191126 20982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786267941944493-20982-0/minicluster-data/master-0/wal/instance
I20260809 09:33:26.191468 20982 external_mini_cluster.cc:1428] Setting key f469e9202975b9a40bd02f49d7fa5238
I20260809 09:33:26.192808 2453 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 09:33:26.198515 2453 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9e13e5aa055847c892299e4c0d1c45d8: Bootstrap starting.
I20260809 09:33:26.200584 2456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:33:26.192898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42880 (local address 127.20.125.190:36357)
0809 09:33:26.193523 (+ 625us) server_negotiation.cc:206] Beginning negotiation
0809 09:33:26.193546 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0809 09:33:26.193580 (+ 34us) server_negotiation.cc:407] Connection header received
0809 09:33:26.194216 (+ 636us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:33:26.194234 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:33:26.194677 (+ 443us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:33:26.195011 (+ 334us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:33:26.195429 (+ 418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:33:26.196373 (+ 944us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:33:26.197280 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:33:26.197752 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:33:26.197942 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:33:26.198911 (+ 969us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:33:26.198917 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:33:26.198928 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 09:33:26.199313 (+ 385us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:33:26.199367 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 09:33:26.199388 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:33:26.199500 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0809 09:33:26.199636 (+ 136us) server_negotiation.cc:299] Negotiation successful
0809 09:33:26.199902 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":290,"thread_start_us":134,"threads_started":1}
I20260809 09:33:26.200937 2453 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9e13e5aa055847c892299e4c0d1c45d8: Neither blocks nor log segments found. Creating new log.
I20260809 09:33:26.201819 2453 log.cc:824] T 00000000000000000000000000000000 P 9e13e5aa055847c892299e4c0d1c45d8: Log is configured to *not* fsync() on all Append() calls
I20260809 09:33:26.204056 2453 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9e13e5aa055847c892299e4c0d1c45d8: No bootstrap required, opened a new log
I20260809 09:33:26.207588 2453 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9e13e5aa055847c892299e4c0d1c45d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e13e5aa055847c892299e4c0d1c45d8" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 36357 } }
I20260809 09:33:26.207809 2453 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9e13e5aa055847c892299e4c0d1c45d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 09:33:26.207894 2453 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9e13e5aa055847c892299e4c0d1c45d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e13e5aa055847c892299e4c0d1c45d8, State: Initialized, Role: FOLLOWER
I20260809 09:33:26.208348 2453 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9e13e5aa055847c892299e4c0d1c45d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e13e5aa055847c892299e4c0d1c45d8" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 36357 } }
I20260809 09:33:26.208499 2453 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9e13e5aa055847c892299e4c0d1c45d8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 09:33:26.208619 2453 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9e13e5aa055847c892299e4c0d1c45d8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 09:33:26.208739 2453 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9e13e5aa055847c892299e4c0d1c45d8 [term 0 FOLLOWER]: Advancing to term 1
I20260809 09:33:26.209784 2453 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9e13e5aa055847c892299e4c0d1c45d8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e13e5aa055847c892299e4c0d1c45d8" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 36357 } }
I20260809 09:33:26.210139 2453 leader_election.cc:302] T 00000000000000000000000000000000 P 9e13e5aa055847c892299e4c0d1c45d8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9e13e5aa055847c892299e4c0d1c45d8; no voters:
I20260809 09:33:26.210448 2453 leader_election.cc:288] T 00000000000000000000000000000000 P 9e13e5aa055847c892299e4c0d1c45d8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 09:33:26.210637 2460 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9e13e5aa055847c892299e4c0d1c45d8 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 09:33:26.210898 2460 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9e13e5aa055847c892299e4c0d1c45d8 [term 1 LEADER]: Becoming Leader. State: Replica: 9e13e5aa055847c892299e4c0d1c45d8, State: Running, Role: LEADER
I20260809 09:33:26.211256 2460 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9e13e5aa055847c892299e4c0d1c45d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e13e5aa055847c892299e4c0d1c45d8" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 36357 } }
I20260809 09:33:26.211701 2453 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9e13e5aa055847c892299e4c0d1c45d8 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 09:33:26.212942 2463 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9e13e5aa055847c892299e4c0d1c45d8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9e13e5aa055847c892299e4c0d1c45d8. Latest consensus state: current_term: 1 leader_uuid: "9e13e5aa055847c892299e4c0d1c45d8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e13e5aa055847c892299e4c0d1c45d8" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 36357 } } }
I20260809 09:33:26.212996 2462 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9e13e5aa055847c892299e4c0d1c45d8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9e13e5aa055847c892299e4c0d1c45d8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e13e5aa055847c892299e4c0d1c45d8" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 36357 } } }
I20260809 09:33:26.213074 2463 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9e13e5aa055847c892299e4c0d1c45d8 [sys.catalog]: This master's current role is: LEADER
I20260809 09:33:26.213101 2462 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9e13e5aa055847c892299e4c0d1c45d8 [sys.catalog]: This master's current role is: LEADER
I20260809 09:33:26.213445 2471 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 09:33:26.216361 2471 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 09:33:26.222044 2471 catalog_manager.cc:1347] Generated new cluster ID: d02cbdc482064ab2bbf1c18a5b61d1d0
I20260809 09:33:26.222129 2471 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 09:33:26.234908 2471 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 09:33:26.235873 2471 catalog_manager.cc:1504] Loading token signing keys...
I20260809 09:33:26.241175 2471 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9e13e5aa055847c892299e4c0d1c45d8: Generated new TSK 0
I20260809 09:33:26.241936 2471 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 09:33:26.250089 20982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
/tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786267941944493-20982-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786267941944493-20982-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786267941944493-20982-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786267941944493-20982-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.125.129:0
--local_ip_for_outbound_sockets=127.20.125.129
--webserver_interface=127.20.125.129
--webserver_port=0
--tserver_master_addrs=127.20.125.190:36357
--builtin_ntp_servers=127.20.125.148:44839
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260809 09:33:26.373983 2484 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 09:33:26.374253 2484 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 09:33:26.374293 2484 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 09:33:26.378921 2484 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 09:33:26.378983 2484 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 09:33:26.379065 2484 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.125.129
I20260809 09:33:26.383821 2484 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.125.148:44839
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786267941944493-20982-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786267941944493-20982-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.125.129:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786267941944493-20982-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.125.129
--webserver_port=0
--tserver_master_addrs=127.20.125.190:36357
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2484
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.125.129
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786267941944493-20982-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 08:38:55 UTC on ccycloud.quasar-njjleu.root.comops.site
build id 4575935
I20260809 09:33:26.384914 2484 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 09:33:26.386312 2484 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 09:33:26.396988 2492 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:33:26.396982 2493 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:33:26.397038 2484 server_base.cc:1034] running on GCE node
W20260809 09:33:26.397218 2495 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:33:26.397714 2484 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 09:33:26.398407 2484 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 09:33:26.399631 2484 hybrid_clock.cc:648] HybridClock initialized: now 1786268006399603 us; error 53 us; skew 500 ppm
I20260809 09:33:26.401489 2484 webserver.cc:460] Webserver started at http://127.20.125.129:45375/ using document root <none> and password file <none>
I20260809 09:33:26.402104 2484 fs_manager.cc:359] Metadata directory not provided
I20260809 09:33:26.402204 2484 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 09:33:26.402455 2484 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 09:33:26.404595 2484 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786267941944493-20982-0/minicluster-data/ts-0/data/instance:
uuid: "a735c3583be54f07806dfb86d3691102"
format_stamp: "Formatted at 2026-08-09 09:33:26 on dist-test-slave-7kzw"
server_key: "b7276dc1e71df8cc72708239ba2971aa"
server_key_iv: "c62283f46972b150c60c75c470e104dd"
server_key_version: "encryptionkey@0"
I20260809 09:33:26.405180 2484 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786267941944493-20982-0/minicluster-data/ts-0/wal/instance:
uuid: "a735c3583be54f07806dfb86d3691102"
format_stamp: "Formatted at 2026-08-09 09:33:26 on dist-test-slave-7kzw"
server_key: "b7276dc1e71df8cc72708239ba2971aa"
server_key_iv: "c62283f46972b150c60c75c470e104dd"
server_key_version: "encryptionkey@0"
I20260809 09:33:26.409096 2484 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.002s
I20260809 09:33:26.411875 2503 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 09:33:26.413436 2484 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 09:33:26.413578 2484 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786267941944493-20982-0/minicluster-data/ts-0/data,/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786267941944493-20982-0/minicluster-data/ts-0/wal
uuid: "a735c3583be54f07806dfb86d3691102"
format_stamp: "Formatted at 2026-08-09 09:33:26 on dist-test-slave-7kzw"
server_key: "b7276dc1e71df8cc72708239ba2971aa"
server_key_iv: "c62283f46972b150c60c75c470e104dd"
server_key_version: "encryptionkey@0"
I20260809 09:33:26.413712 2484 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786267941944493-20982-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786267941944493-20982-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786267941944493-20982-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 09:33:26.431586 2484 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 09:33:26.432312 2484 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 09:33:26.432523 2484 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 09:33:26.433140 2484 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 09:33:26.434149 2484 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 09:33:26.434226 2484 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 09:33:26.434298 2484 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 09:33:26.434350 2484 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 09:33:26.443969 2484 rpc_server.cc:307] RPC server started. Bound to: 127.20.125.129:43719
I20260809 09:33:26.443987 2660 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.125.129:43719 every 8 connection(s)
I20260809 09:33:26.444942 2484 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786267941944493-20982-0/minicluster-data/ts-0/data/info.pb
I20260809 09:33:26.448879 20982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu as pid 2484
I20260809 09:33:26.449018 20982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786267941944493-20982-0/minicluster-data/ts-0/wal/instance
I20260809 09:33:26.449330 20982 external_mini_cluster.cc:1428] Setting key 9d0d47ebcd37d2e6585aa81390035b80
I20260809 09:33:26.451793 20982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
/tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786267941944493-20982-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786267941944493-20982-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786267941944493-20982-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786267941944493-20982-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.125.130:0
--local_ip_for_outbound_sockets=127.20.125.130
--webserver_interface=127.20.125.130
--webserver_port=0
--tserver_master_addrs=127.20.125.190:36357
--builtin_ntp_servers=127.20.125.148:44839
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260809 09:33:26.455842 2456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:33:26.446914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.129:40303 (local address 127.20.125.190:36357)
0809 09:33:26.447067 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0809 09:33:26.447071 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 09:33:26.448244 (+ 1173us) server_negotiation.cc:407] Connection header received
0809 09:33:26.449988 (+ 1744us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:33:26.449992 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:33:26.450057 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:33:26.450178 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:33:26.451346 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:33:26.452249 (+ 903us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:33:26.453491 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:33:26.453919 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:33:26.454056 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:33:26.455204 (+ 1148us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:33:26.455209 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:33:26.455213 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 09:33:26.455235 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:33:26.455266 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 09:33:26.455280 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:33:26.455369 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0809 09:33:26.455530 (+ 161us) server_negotiation.cc:299] Negotiation successful
0809 09:33:26.455613 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260809 09:33:26.456594 2664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:33:26.447362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:36357 (local address 127.20.125.129:40303)
0809 09:33:26.448055 (+ 693us) negotiation.cc:107] Waiting for socket to connect
0809 09:33:26.448099 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 09:33:26.449611 (+ 1512us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:33:26.450385 (+ 774us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:33:26.450394 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:33:26.450521 (+ 127us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:33:26.451178 (+ 657us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:33:26.451201 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:33:26.452387 (+ 1186us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:33:26.452389 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:33:26.453341 (+ 952us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:33:26.453354 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:33:26.454080 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:33:26.454082 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:33:26.454236 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:33:26.454919 (+ 683us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 09:33:26.454920 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:33:26.454959 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 09:33:26.455024 (+ 65us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 09:33:26.455383 (+ 359us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 09:33:26.455391 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 09:33:26.455395 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 09:33:26.455486 (+ 91us) client_negotiation.cc:240] Negotiation successful
0809 09:33:26.455714 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":285,"thread_start_us":120,"threads_started":1}
I20260809 09:33:26.457976 2662 heartbeater.cc:347] Connected to a master server at 127.20.125.190:36357
I20260809 09:33:26.458212 2662 heartbeater.cc:464] Registering TS with master...
I20260809 09:33:26.458794 2662 heartbeater.cc:511] Master 127.20.125.190:36357 requested a full tablet report, sending...
I20260809 09:33:26.460326 2403 ts_manager.cc:194] Registered new tserver with Master: a735c3583be54f07806dfb86d3691102 (127.20.125.129:43719)
I20260809 09:33:26.461459 2403 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.20.125.129:40303
W20260809 09:33:26.575538 2665 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 09:33:26.575809 2665 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 09:33:26.575848 2665 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 09:33:26.579877 2665 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 09:33:26.579929 2665 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 09:33:26.580013 2665 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.125.130
I20260809 09:33:26.584723 2665 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.125.148:44839
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786267941944493-20982-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786267941944493-20982-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.125.130:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786267941944493-20982-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.125.130
--webserver_port=0
--tserver_master_addrs=127.20.125.190:36357
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2665
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.125.130
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786267941944493-20982-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 08:38:55 UTC on ccycloud.quasar-njjleu.root.comops.site
build id 4575935
I20260809 09:33:26.585819 2665 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 09:33:26.587245 2665 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 09:33:26.597095 2674 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:33:26.597087 2677 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:33:26.597087 2675 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:33:26.597473 2665 server_base.cc:1034] running on GCE node
I20260809 09:33:26.597882 2665 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 09:33:26.598558 2665 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 09:33:26.599766 2665 hybrid_clock.cc:648] HybridClock initialized: now 1786268006599749 us; error 31 us; skew 500 ppm
I20260809 09:33:26.601718 2665 webserver.cc:460] Webserver started at http://127.20.125.130:43477/ using document root <none> and password file <none>
I20260809 09:33:26.602319 2665 fs_manager.cc:359] Metadata directory not provided
I20260809 09:33:26.602415 2665 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 09:33:26.602681 2665 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 09:33:26.604733 2665 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786267941944493-20982-0/minicluster-data/ts-1/data/instance:
uuid: "2b649194a49f41ba96f090a52121d5b9"
format_stamp: "Formatted at 2026-08-09 09:33:26 on dist-test-slave-7kzw"
server_key: "345a02ae4d28efb1cb14ccfba96c3718"
server_key_iv: "85b2496164c2a9c59c9f4fd9bace76d8"
server_key_version: "encryptionkey@0"
I20260809 09:33:26.605309 2665 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786267941944493-20982-0/minicluster-data/ts-1/wal/instance:
uuid: "2b649194a49f41ba96f090a52121d5b9"
format_stamp: "Formatted at 2026-08-09 09:33:26 on dist-test-slave-7kzw"
server_key: "345a02ae4d28efb1cb14ccfba96c3718"
server_key_iv: "85b2496164c2a9c59c9f4fd9bace76d8"
server_key_version: "encryptionkey@0"
I20260809 09:33:26.609038 2665 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260809 09:33:26.611857 2685 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 09:33:26.613404 2665 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 09:33:26.613584 2665 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786267941944493-20982-0/minicluster-data/ts-1/data,/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786267941944493-20982-0/minicluster-data/ts-1/wal
uuid: "2b649194a49f41ba96f090a52121d5b9"
format_stamp: "Formatted at 2026-08-09 09:33:26 on dist-test-slave-7kzw"
server_key: "345a02ae4d28efb1cb14ccfba96c3718"
server_key_iv: "85b2496164c2a9c59c9f4fd9bace76d8"
server_key_version: "encryptionkey@0"
I20260809 09:33:26.613689 2665 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786267941944493-20982-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786267941944493-20982-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786267941944493-20982-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 09:33:26.631197 2665 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 09:33:26.631841 2665 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 09:33:26.631994 2665 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 09:33:26.632514 2665 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 09:33:26.633410 2665 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 09:33:26.633460 2665 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 09:33:26.633503 2665 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 09:33:26.633518 2665 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 09:33:26.643067 2665 rpc_server.cc:307] RPC server started. Bound to: 127.20.125.130:33703
I20260809 09:33:26.643158 2856 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.125.130:33703 every 8 connection(s)
I20260809 09:33:26.643970 2665 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786267941944493-20982-0/minicluster-data/ts-1/data/info.pb
I20260809 09:33:26.650830 20982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu as pid 2665
I20260809 09:33:26.650955 20982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786267941944493-20982-0/minicluster-data/ts-1/wal/instance
I20260809 09:33:26.651315 20982 external_mini_cluster.cc:1428] Setting key 1e7028846702c59be13ee6d183461d32
I20260809 09:33:26.653506 20982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
/tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786267941944493-20982-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786267941944493-20982-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786267941944493-20982-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786267941944493-20982-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.125.131:0
--local_ip_for_outbound_sockets=127.20.125.131
--webserver_interface=127.20.125.131
--webserver_port=0
--tserver_master_addrs=127.20.125.190:36357
--builtin_ntp_servers=127.20.125.148:44839
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260809 09:33:26.654412 2456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:33:26.645942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.130:34475 (local address 127.20.125.190:36357)
0809 09:33:26.646116 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0809 09:33:26.646120 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 09:33:26.647302 (+ 1182us) server_negotiation.cc:407] Connection header received
0809 09:33:26.648789 (+ 1487us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:33:26.648793 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:33:26.648840 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:33:26.648916 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:33:26.649970 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:33:26.650545 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:33:26.652093 (+ 1548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:33:26.652338 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:33:26.652465 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:33:26.653796 (+ 1331us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:33:26.653805 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:33:26.653807 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 09:33:26.653819 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:33:26.653838 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 09:33:26.653846 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:33:26.653924 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0809 09:33:26.654114 (+ 190us) server_negotiation.cc:299] Negotiation successful
0809 09:33:26.654192 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260809 09:33:26.655323 2859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:33:26.646324 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:36357 (local address 127.20.125.130:34475)
0809 09:33:26.647132 (+ 808us) negotiation.cc:107] Waiting for socket to connect
0809 09:33:26.647176 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 09:33:26.648586 (+ 1410us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:33:26.649170 (+ 584us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:33:26.649178 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:33:26.649268 (+ 90us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:33:26.649824 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:33:26.649840 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:33:26.650701 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:33:26.650705 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:33:26.651976 (+ 1271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:33:26.651987 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:33:26.652510 (+ 523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:33:26.652514 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:33:26.652776 (+ 262us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:33:26.653478 (+ 702us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 09:33:26.653480 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:33:26.653520 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 09:33:26.653625 (+ 105us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 09:33:26.653928 (+ 303us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 09:33:26.653940 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 09:33:26.653947 (+ 7us) client_negotiation.cc:769] Sending connection context
0809 09:33:26.654072 (+ 125us) client_negotiation.cc:240] Negotiation successful
0809 09:33:26.654375 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":389,"thread_start_us":172,"threads_started":1}
I20260809 09:33:26.656543 2857 heartbeater.cc:347] Connected to a master server at 127.20.125.190:36357
I20260809 09:33:26.656808 2857 heartbeater.cc:464] Registering TS with master...
I20260809 09:33:26.657423 2857 heartbeater.cc:511] Master 127.20.125.190:36357 requested a full tablet report, sending...
I20260809 09:33:26.658730 2403 ts_manager.cc:194] Registered new tserver with Master: 2b649194a49f41ba96f090a52121d5b9 (127.20.125.130:33703)
I20260809 09:33:26.659621 2403 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.20.125.130:34475
W20260809 09:33:26.774048 2860 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 09:33:26.774348 2860 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 09:33:26.774389 2860 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 09:33:26.778568 2860 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 09:33:26.778620 2860 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 09:33:26.778678 2860 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.125.131
I20260809 09:33:26.783321 2860 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.125.148:44839
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786267941944493-20982-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786267941944493-20982-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.125.131:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786267941944493-20982-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.125.131
--webserver_port=0
--tserver_master_addrs=127.20.125.190:36357
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2860
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.125.131
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786267941944493-20982-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 08:38:55 UTC on ccycloud.quasar-njjleu.root.comops.site
build id 4575935
I20260809 09:33:26.784389 2860 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 09:33:26.785940 2860 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 09:33:26.796165 2866 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:33:26.796165 2867 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:33:26.796427 2872 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:33:26.796216 2860 server_base.cc:1034] running on GCE node
I20260809 09:33:26.796864 2860 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 09:33:26.797457 2860 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 09:33:26.798652 2860 hybrid_clock.cc:648] HybridClock initialized: now 1786268006798632 us; error 31 us; skew 500 ppm
I20260809 09:33:26.800710 2860 webserver.cc:460] Webserver started at http://127.20.125.131:38813/ using document root <none> and password file <none>
I20260809 09:33:26.801364 2860 fs_manager.cc:359] Metadata directory not provided
I20260809 09:33:26.801445 2860 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 09:33:26.801680 2860 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 09:33:26.804011 2860 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786267941944493-20982-0/minicluster-data/ts-2/data/instance:
uuid: "ed5276f341634cae911927107ad398af"
format_stamp: "Formatted at 2026-08-09 09:33:26 on dist-test-slave-7kzw"
server_key: "6c2adcbbe0aaae2a836617ad22296b5d"
server_key_iv: "1b9feac46da0ddf9671380f6d99fb5b0"
server_key_version: "encryptionkey@0"
I20260809 09:33:26.804625 2860 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786267941944493-20982-0/minicluster-data/ts-2/wal/instance:
uuid: "ed5276f341634cae911927107ad398af"
format_stamp: "Formatted at 2026-08-09 09:33:26 on dist-test-slave-7kzw"
server_key: "6c2adcbbe0aaae2a836617ad22296b5d"
server_key_iv: "1b9feac46da0ddf9671380f6d99fb5b0"
server_key_version: "encryptionkey@0"
I20260809 09:33:26.808755 2860 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 09:33:26.811489 2881 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 09:33:26.813249 2860 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 09:33:26.813365 2860 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786267941944493-20982-0/minicluster-data/ts-2/data,/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786267941944493-20982-0/minicluster-data/ts-2/wal
uuid: "ed5276f341634cae911927107ad398af"
format_stamp: "Formatted at 2026-08-09 09:33:26 on dist-test-slave-7kzw"
server_key: "6c2adcbbe0aaae2a836617ad22296b5d"
server_key_iv: "1b9feac46da0ddf9671380f6d99fb5b0"
server_key_version: "encryptionkey@0"
I20260809 09:33:26.813467 2860 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786267941944493-20982-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786267941944493-20982-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786267941944493-20982-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 09:33:26.837901 2860 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 09:33:26.838691 2860 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 09:33:26.838893 2860 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 09:33:26.839497 2860 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 09:33:26.840456 2860 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 09:33:26.840503 2860 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 09:33:26.840579 2860 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 09:33:26.840618 2860 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 09:33:26.849440 2860 rpc_server.cc:307] RPC server started. Bound to: 127.20.125.131:40801
I20260809 09:33:26.849471 3049 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.125.131:40801 every 8 connection(s)
I20260809 09:33:26.850365 2860 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786267941944493-20982-0/minicluster-data/ts-2/data/info.pb
I20260809 09:33:26.851969 20982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu as pid 2860
I20260809 09:33:26.852061 20982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786267941944493-20982-0/minicluster-data/ts-2/wal/instance
I20260809 09:33:26.852442 20982 external_mini_cluster.cc:1428] Setting key 4600f691ca808400a94c3d8708034177
I20260809 09:33:26.860363 2456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:33:26.852396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.131:51425 (local address 127.20.125.190:36357)
0809 09:33:26.852563 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0809 09:33:26.852568 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 09:33:26.853841 (+ 1273us) server_negotiation.cc:407] Connection header received
0809 09:33:26.855525 (+ 1684us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:33:26.855528 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:33:26.855576 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:33:26.855680 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:33:26.856563 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:33:26.857117 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:33:26.858279 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:33:26.858609 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:33:26.858713 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:33:26.859815 (+ 1102us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:33:26.859818 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:33:26.859820 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 09:33:26.859832 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:33:26.859855 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 09:33:26.859864 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:33:26.859941 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0809 09:33:26.860181 (+ 240us) server_negotiation.cc:299] Negotiation successful
0809 09:33:26.860247 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260809 09:33:26.861275 3052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:33:26.852926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:36357 (local address 127.20.125.131:51425)
0809 09:33:26.853691 (+ 765us) negotiation.cc:107] Waiting for socket to connect
0809 09:33:26.853744 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0809 09:33:26.855326 (+ 1582us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:33:26.855840 (+ 514us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:33:26.855846 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:33:26.855927 (+ 81us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:33:26.856429 (+ 502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:33:26.856442 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:33:26.857267 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:33:26.857270 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:33:26.858143 (+ 873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:33:26.858152 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:33:26.858775 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:33:26.858778 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:33:26.858932 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:33:26.859526 (+ 594us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 09:33:26.859528 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:33:26.859568 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 09:33:26.859632 (+ 64us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 09:33:26.859984 (+ 352us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 09:33:26.859994 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 09:33:26.859999 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 09:33:26.860135 (+ 136us) client_negotiation.cc:240] Negotiation successful
0809 09:33:26.860367 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":307,"thread_start_us":145,"threads_started":1}
I20260809 09:33:26.862701 3050 heartbeater.cc:347] Connected to a master server at 127.20.125.190:36357
I20260809 09:33:26.862959 3050 heartbeater.cc:464] Registering TS with master...
I20260809 09:33:26.863507 3050 heartbeater.cc:511] Master 127.20.125.190:36357 requested a full tablet report, sending...
I20260809 09:33:26.864703 2403 ts_manager.cc:194] Registered new tserver with Master: ed5276f341634cae911927107ad398af (127.20.125.131:40801)
I20260809 09:33:26.865329 2403 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.20.125.131:51425
I20260809 09:33:26.866750 20982 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260809 09:33:26.869508 2456 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 09:33:26.868760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42888 (local address 127.20.125.190:36357)
0809 09:33:26.868885 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0809 09:33:26.868888 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 09:33:26.869013 (+ 125us) server_negotiation.cc:407] Connection header received
0809 09:33:26.869110 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:33:26.869112 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:33:26.869147 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:33:26.869226 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:33:26.869413 (+ 187us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42888: BlockingRecv error: recv got EOF from 127.0.0.1:42888 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
I20260809 09:33:26.870071 20982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu with pid 2484
I20260809 09:33:26.881518 20982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu with pid 2665
I20260809 09:33:26.898801 20982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu with pid 2860
I20260809 09:33:26.910413 20982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu with pid 2359
2026-08-09T09:33:26Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (984 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-09T09:33:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T09:33:26Z Disabled control of system clock
I20260809 09:33:26.947304 20982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
/tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786267941944493-20982-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786267941944493-20982-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786267941944493-20982-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786267941944493-20982-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.125.190:39021
--webserver_interface=127.20.125.190
--webserver_port=0
--builtin_ntp_servers=127.20.125.148:35005
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.125.190:39021
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260809 09:33:27.061717 3065 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 09:33:27.062011 3065 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 09:33:27.062049 3065 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 09:33:27.066103 3065 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 09:33:27.066179 3065 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 09:33:27.066195 3065 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 09:33:27.066212 3065 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 09:33:27.066227 3065 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 09:33:27.071043 3065 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.125.148:35005
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786267941944493-20982-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786267941944493-20982-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.125.190:39021
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.125.190:39021
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786267941944493-20982-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.125.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3065
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786267941944493-20982-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 08:38:55 UTC on ccycloud.quasar-njjleu.root.comops.site
build id 4575935
I20260809 09:33:27.072144 3065 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 09:33:27.073666 3065 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 09:33:27.082541 3075 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:33:27.082538 3071 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:33:27.082535 3072 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:33:27.082823 3065 server_base.cc:1034] running on GCE node
I20260809 09:33:27.083451 3065 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 09:33:27.084583 3065 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 09:33:27.085813 3065 hybrid_clock.cc:648] HybridClock initialized: now 1786268007085772 us; error 50 us; skew 500 ppm
I20260809 09:33:27.088068 3065 webserver.cc:460] Webserver started at http://127.20.125.190:42123/ using document root <none> and password file <none>
I20260809 09:33:27.088701 3065 fs_manager.cc:359] Metadata directory not provided
I20260809 09:33:27.088807 3065 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 09:33:27.089068 3065 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 09:33:27.091325 3065 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786267941944493-20982-0/minicluster-data/master-0/data/instance:
uuid: "056a331f2d9a44cb8aa9f0c34f25fab3"
format_stamp: "Formatted at 2026-08-09 09:33:27 on dist-test-slave-7kzw"
server_key: "8f141d0415b6d22f28a5e76bbc3cae0a"
server_key_iv: "5790179023fe123a85d991799c811f5e"
server_key_version: "encryptionkey@0"
I20260809 09:33:27.091936 3065 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786267941944493-20982-0/minicluster-data/master-0/wal/instance:
uuid: "056a331f2d9a44cb8aa9f0c34f25fab3"
format_stamp: "Formatted at 2026-08-09 09:33:27 on dist-test-slave-7kzw"
server_key: "8f141d0415b6d22f28a5e76bbc3cae0a"
server_key_iv: "5790179023fe123a85d991799c811f5e"
server_key_version: "encryptionkey@0"
I20260809 09:33:27.095854 3065 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260809 09:33:27.098368 3082 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 09:33:27.099746 3065 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 09:33:27.099900 3065 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786267941944493-20982-0/minicluster-data/master-0/data,/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786267941944493-20982-0/minicluster-data/master-0/wal
uuid: "056a331f2d9a44cb8aa9f0c34f25fab3"
format_stamp: "Formatted at 2026-08-09 09:33:27 on dist-test-slave-7kzw"
server_key: "8f141d0415b6d22f28a5e76bbc3cae0a"
server_key_iv: "5790179023fe123a85d991799c811f5e"
server_key_version: "encryptionkey@0"
I20260809 09:33:27.100020 3065 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786267941944493-20982-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786267941944493-20982-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786267941944493-20982-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 09:33:27.104696 3065 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 09:33:27.105314 3065 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 09:33:27.105499 3065 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 09:33:27.112811 3065 rpc_server.cc:307] RPC server started. Bound to: 127.20.125.190:39021
I20260809 09:33:27.112823 3159 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.125.190:39021 every 8 connection(s)
I20260809 09:33:27.113730 3065 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786267941944493-20982-0/minicluster-data/master-0/data/info.pb
I20260809 09:33:27.117110 3160 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 09:33:27.122728 3160 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 056a331f2d9a44cb8aa9f0c34f25fab3: Bootstrap starting.
I20260809 09:33:27.123808 20982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu as pid 3065
I20260809 09:33:27.123953 20982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786267941944493-20982-0/minicluster-data/master-0/wal/instance
I20260809 09:33:27.124300 20982 external_mini_cluster.cc:1428] Setting key a53e372e3f9cf805028fcd4196168420
I20260809 09:33:27.125368 3160 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 056a331f2d9a44cb8aa9f0c34f25fab3: Neither blocks nor log segments found. Creating new log.
I20260809 09:33:27.126230 3160 log.cc:824] T 00000000000000000000000000000000 P 056a331f2d9a44cb8aa9f0c34f25fab3: Log is configured to *not* fsync() on all Append() calls
I20260809 09:33:27.128346 3160 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 056a331f2d9a44cb8aa9f0c34f25fab3: No bootstrap required, opened a new log
I20260809 09:33:27.130170 3163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:33:27.125567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35436 (local address 127.20.125.190:39021)
0809 09:33:27.126258 (+ 691us) server_negotiation.cc:206] Beginning negotiation
0809 09:33:27.126275 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0809 09:33:27.126340 (+ 65us) server_negotiation.cc:407] Connection header received
0809 09:33:27.127301 (+ 961us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:33:27.127323 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:33:27.127614 (+ 291us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:33:27.127971 (+ 357us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:33:27.128885 (+ 914us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:33:27.128890 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:33:27.128901 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 09:33:27.128913 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:33:27.128974 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 09:33:27.128992 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:33:27.129124 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0809 09:33:27.129244 (+ 120us) server_negotiation.cc:299] Negotiation successful
0809 09:33:27.129489 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":314,"thread_start_us":135,"threads_started":1}
I20260809 09:33:27.132023 3160 raft_consensus.cc:348] T 00000000000000000000000000000000 P 056a331f2d9a44cb8aa9f0c34f25fab3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "056a331f2d9a44cb8aa9f0c34f25fab3" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 39021 } }
I20260809 09:33:27.132283 3160 raft_consensus.cc:374] T 00000000000000000000000000000000 P 056a331f2d9a44cb8aa9f0c34f25fab3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 09:33:27.132375 3160 raft_consensus.cc:729] T 00000000000000000000000000000000 P 056a331f2d9a44cb8aa9f0c34f25fab3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 056a331f2d9a44cb8aa9f0c34f25fab3, State: Initialized, Role: FOLLOWER
I20260809 09:33:27.132776 3160 consensus_queue.cc:260] T 00000000000000000000000000000000 P 056a331f2d9a44cb8aa9f0c34f25fab3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "056a331f2d9a44cb8aa9f0c34f25fab3" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 39021 } }
I20260809 09:33:27.132895 3160 raft_consensus.cc:388] T 00000000000000000000000000000000 P 056a331f2d9a44cb8aa9f0c34f25fab3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 09:33:27.132941 3160 raft_consensus.cc:482] T 00000000000000000000000000000000 P 056a331f2d9a44cb8aa9f0c34f25fab3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 09:33:27.133010 3160 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 056a331f2d9a44cb8aa9f0c34f25fab3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 09:33:27.133956 3160 raft_consensus.cc:504] T 00000000000000000000000000000000 P 056a331f2d9a44cb8aa9f0c34f25fab3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "056a331f2d9a44cb8aa9f0c34f25fab3" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 39021 } }
I20260809 09:33:27.134325 3160 leader_election.cc:302] T 00000000000000000000000000000000 P 056a331f2d9a44cb8aa9f0c34f25fab3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 056a331f2d9a44cb8aa9f0c34f25fab3; no voters:
I20260809 09:33:27.134666 3160 leader_election.cc:288] T 00000000000000000000000000000000 P 056a331f2d9a44cb8aa9f0c34f25fab3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 09:33:27.134814 3165 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 056a331f2d9a44cb8aa9f0c34f25fab3 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 09:33:27.135098 3165 raft_consensus.cc:686] T 00000000000000000000000000000000 P 056a331f2d9a44cb8aa9f0c34f25fab3 [term 1 LEADER]: Becoming Leader. State: Replica: 056a331f2d9a44cb8aa9f0c34f25fab3, State: Running, Role: LEADER
I20260809 09:33:27.135409 3165 consensus_queue.cc:237] T 00000000000000000000000000000000 P 056a331f2d9a44cb8aa9f0c34f25fab3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "056a331f2d9a44cb8aa9f0c34f25fab3" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 39021 } }
I20260809 09:33:27.135887 3160 sys_catalog.cc:564] T 00000000000000000000000000000000 P 056a331f2d9a44cb8aa9f0c34f25fab3 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 09:33:27.137148 3166 sys_catalog.cc:455] T 00000000000000000000000000000000 P 056a331f2d9a44cb8aa9f0c34f25fab3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "056a331f2d9a44cb8aa9f0c34f25fab3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "056a331f2d9a44cb8aa9f0c34f25fab3" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 39021 } } }
I20260809 09:33:27.137315 3166 sys_catalog.cc:458] T 00000000000000000000000000000000 P 056a331f2d9a44cb8aa9f0c34f25fab3 [sys.catalog]: This master's current role is: LEADER
I20260809 09:33:27.137148 3167 sys_catalog.cc:455] T 00000000000000000000000000000000 P 056a331f2d9a44cb8aa9f0c34f25fab3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 056a331f2d9a44cb8aa9f0c34f25fab3. Latest consensus state: current_term: 1 leader_uuid: "056a331f2d9a44cb8aa9f0c34f25fab3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "056a331f2d9a44cb8aa9f0c34f25fab3" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 39021 } } }
I20260809 09:33:27.137382 3167 sys_catalog.cc:458] T 00000000000000000000000000000000 P 056a331f2d9a44cb8aa9f0c34f25fab3 [sys.catalog]: This master's current role is: LEADER
I20260809 09:33:27.137732 3181 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 09:33:27.140684 3181 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 09:33:27.146459 3181 catalog_manager.cc:1347] Generated new cluster ID: 1b3fb9da484d48d2a0752362011bf2f7
I20260809 09:33:27.146570 3181 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 09:33:27.168926 3181 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 09:33:27.169310 3181 catalog_manager.cc:1504] Loading token signing keys...
I20260809 09:33:27.175518 3181 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 056a331f2d9a44cb8aa9f0c34f25fab3: Generated new TSK 0
I20260809 09:33:27.176276 3181 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 09:33:27.179450 20982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
/tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786267941944493-20982-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786267941944493-20982-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786267941944493-20982-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786267941944493-20982-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.125.129:0
--local_ip_for_outbound_sockets=127.20.125.129
--webserver_interface=127.20.125.129
--webserver_port=0
--tserver_master_addrs=127.20.125.190:39021
--builtin_ntp_servers=127.20.125.148:35005
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260809 09:33:27.297122 3197 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 09:33:27.297395 3197 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 09:33:27.297467 3197 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 09:33:27.301496 3197 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 09:33:27.301555 3197 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 09:33:27.301656 3197 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.125.129
I20260809 09:33:27.306694 3197 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.125.148:35005
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786267941944493-20982-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786267941944493-20982-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.125.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786267941944493-20982-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.125.129
--webserver_port=0
--tserver_master_addrs=127.20.125.190:39021
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3197
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.125.129
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786267941944493-20982-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 08:38:55 UTC on ccycloud.quasar-njjleu.root.comops.site
build id 4575935
I20260809 09:33:27.307837 3197 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 09:33:27.309327 3197 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 09:33:27.319186 3206 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:33:27.319236 3209 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:33:27.319236 3207 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:33:27.319765 3197 server_base.cc:1034] running on GCE node
I20260809 09:33:27.320214 3197 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 09:33:27.320946 3197 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 09:33:27.322141 3197 hybrid_clock.cc:648] HybridClock initialized: now 1786268007322113 us; error 44 us; skew 500 ppm
I20260809 09:33:27.324029 3197 webserver.cc:460] Webserver started at http://127.20.125.129:36353/ using document root <none> and password file <none>
I20260809 09:33:27.324625 3197 fs_manager.cc:359] Metadata directory not provided
I20260809 09:33:27.324725 3197 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 09:33:27.324970 3197 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 09:33:27.327164 3197 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786267941944493-20982-0/minicluster-data/ts-0/data/instance:
uuid: "3f041820549d47539c05da8eea7392c5"
format_stamp: "Formatted at 2026-08-09 09:33:27 on dist-test-slave-7kzw"
server_key: "223312f8e111c35db6361717a263b9d9"
server_key_iv: "f91c29c328c353d21d09440fbfa8f507"
server_key_version: "encryptionkey@0"
I20260809 09:33:27.327781 3197 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786267941944493-20982-0/minicluster-data/ts-0/wal/instance:
uuid: "3f041820549d47539c05da8eea7392c5"
format_stamp: "Formatted at 2026-08-09 09:33:27 on dist-test-slave-7kzw"
server_key: "223312f8e111c35db6361717a263b9d9"
server_key_iv: "f91c29c328c353d21d09440fbfa8f507"
server_key_version: "encryptionkey@0"
I20260809 09:33:27.331643 3197 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260809 09:33:27.334415 3218 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 09:33:27.335886 3197 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 09:33:27.336027 3197 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786267941944493-20982-0/minicluster-data/ts-0/data,/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786267941944493-20982-0/minicluster-data/ts-0/wal
uuid: "3f041820549d47539c05da8eea7392c5"
format_stamp: "Formatted at 2026-08-09 09:33:27 on dist-test-slave-7kzw"
server_key: "223312f8e111c35db6361717a263b9d9"
server_key_iv: "f91c29c328c353d21d09440fbfa8f507"
server_key_version: "encryptionkey@0"
I20260809 09:33:27.336146 3197 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786267941944493-20982-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786267941944493-20982-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786267941944493-20982-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 09:33:27.341329 3197 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 09:33:27.341957 3197 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 09:33:27.342166 3197 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 09:33:27.342782 3197 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 09:33:27.343710 3197 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 09:33:27.343783 3197 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 09:33:27.343856 3197 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 09:33:27.343899 3197 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 09:33:27.353499 3197 rpc_server.cc:307] RPC server started. Bound to: 127.20.125.129:38915
I20260809 09:33:27.353521 3398 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.125.129:38915 every 8 connection(s)
I20260809 09:33:27.354413 3197 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786267941944493-20982-0/minicluster-data/ts-0/data/info.pb
I20260809 09:33:27.357573 20982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu as pid 3197
I20260809 09:33:27.357734 20982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786267941944493-20982-0/minicluster-data/ts-0/wal/instance
I20260809 09:33:27.358115 20982 external_mini_cluster.cc:1428] Setting key 081938d2cb3be9779c1c3d3d884993f3
I20260809 09:33:27.360671 20982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
/tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786267941944493-20982-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786267941944493-20982-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786267941944493-20982-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786267941944493-20982-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.125.130:0
--local_ip_for_outbound_sockets=127.20.125.130
--webserver_interface=127.20.125.130
--webserver_port=0
--tserver_master_addrs=127.20.125.190:39021
--builtin_ntp_servers=127.20.125.148:35005
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260809 09:33:27.361498 3163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:33:27.356509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.129:60911 (local address 127.20.125.190:39021)
0809 09:33:27.356683 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0809 09:33:27.356687 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 09:33:27.357816 (+ 1129us) server_negotiation.cc:407] Connection header received
0809 09:33:27.359376 (+ 1560us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:33:27.359379 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:33:27.359419 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:33:27.359504 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:33:27.360868 (+ 1364us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:33:27.360874 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:33:27.360879 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 09:33:27.360896 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:33:27.360923 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 09:33:27.360936 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:33:27.361012 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 09:33:27.361254 (+ 242us) server_negotiation.cc:299] Negotiation successful
0809 09:33:27.361305 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260809 09:33:27.363087 3404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:33:27.356925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:39021 (local address 127.20.125.129:60911)
0809 09:33:27.357646 (+ 721us) negotiation.cc:107] Waiting for socket to connect
0809 09:33:27.357698 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0809 09:33:27.359171 (+ 1473us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:33:27.359681 (+ 510us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:33:27.359688 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:33:27.359796 (+ 108us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:33:27.360498 (+ 702us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 09:33:27.360502 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:33:27.360589 (+ 87us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 09:33:27.360680 (+ 91us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 09:33:27.361090 (+ 410us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 09:33:27.361101 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 09:33:27.361109 (+ 8us) client_negotiation.cc:769] Sending connection context
0809 09:33:27.361781 (+ 672us) client_negotiation.cc:240] Negotiation successful
0809 09:33:27.362025 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":274,"thread_start_us":146,"threads_started":1}
I20260809 09:33:27.364494 3401 heartbeater.cc:347] Connected to a master server at 127.20.125.190:39021
I20260809 09:33:27.364733 3401 heartbeater.cc:464] Registering TS with master...
I20260809 09:33:27.365142 3401 heartbeater.cc:511] Master 127.20.125.190:39021 requested a full tablet report, sending...
I20260809 09:33:27.366704 3103 ts_manager.cc:194] Registered new tserver with Master: 3f041820549d47539c05da8eea7392c5 (127.20.125.129:38915)
W20260809 09:33:27.480958 3405 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 09:33:27.481256 3405 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 09:33:27.481330 3405 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 09:33:27.485472 3405 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 09:33:27.485564 3405 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 09:33:27.485666 3405 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.125.130
I20260809 09:33:27.490701 3405 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.125.148:35005
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786267941944493-20982-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786267941944493-20982-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.125.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786267941944493-20982-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.125.130
--webserver_port=0
--tserver_master_addrs=127.20.125.190:39021
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3405
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.125.130
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786267941944493-20982-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 08:38:55 UTC on ccycloud.quasar-njjleu.root.comops.site
build id 4575935
I20260809 09:33:27.491891 3405 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 09:33:27.493325 3405 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 09:33:27.503312 3412 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:33:27.503333 3414 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:33:27.503422 3405 server_base.cc:1034] running on GCE node
W20260809 09:33:27.503305 3416 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:33:27.503989 3405 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 09:33:27.504655 3405 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 09:33:27.505829 3405 hybrid_clock.cc:648] HybridClock initialized: now 1786268007505814 us; error 25 us; skew 500 ppm
I20260809 09:33:27.507872 3405 webserver.cc:460] Webserver started at http://127.20.125.130:35739/ using document root <none> and password file <none>
I20260809 09:33:27.508477 3405 fs_manager.cc:359] Metadata directory not provided
I20260809 09:33:27.508546 3405 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 09:33:27.508783 3405 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 09:33:27.510962 3405 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786267941944493-20982-0/minicluster-data/ts-1/data/instance:
uuid: "2cdaef2ee29b420abfcf700b002d7716"
format_stamp: "Formatted at 2026-08-09 09:33:27 on dist-test-slave-7kzw"
server_key: "998ac901f0392862dba8d9f701cfdee4"
server_key_iv: "f970bbcfde18d1426673518f34493851"
server_key_version: "encryptionkey@0"
I20260809 09:33:27.511529 3405 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786267941944493-20982-0/minicluster-data/ts-1/wal/instance:
uuid: "2cdaef2ee29b420abfcf700b002d7716"
format_stamp: "Formatted at 2026-08-09 09:33:27 on dist-test-slave-7kzw"
server_key: "998ac901f0392862dba8d9f701cfdee4"
server_key_iv: "f970bbcfde18d1426673518f34493851"
server_key_version: "encryptionkey@0"
I20260809 09:33:27.515359 3405 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260809 09:33:27.517846 3424 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 09:33:27.519150 3405 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260809 09:33:27.519274 3405 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786267941944493-20982-0/minicluster-data/ts-1/data,/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786267941944493-20982-0/minicluster-data/ts-1/wal
uuid: "2cdaef2ee29b420abfcf700b002d7716"
format_stamp: "Formatted at 2026-08-09 09:33:27 on dist-test-slave-7kzw"
server_key: "998ac901f0392862dba8d9f701cfdee4"
server_key_iv: "f970bbcfde18d1426673518f34493851"
server_key_version: "encryptionkey@0"
I20260809 09:33:27.519380 3405 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786267941944493-20982-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786267941944493-20982-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786267941944493-20982-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 09:33:27.523882 3405 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 09:33:27.524492 3405 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 09:33:27.524672 3405 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 09:33:27.525215 3405 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 09:33:27.526117 3405 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 09:33:27.526165 3405 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 09:33:27.526203 3405 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 09:33:27.526263 3405 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 09:33:27.535969 3405 rpc_server.cc:307] RPC server started. Bound to: 127.20.125.130:39041
I20260809 09:33:27.536021 3589 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.125.130:39041 every 8 connection(s)
I20260809 09:33:27.537022 3405 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786267941944493-20982-0/minicluster-data/ts-1/data/info.pb
I20260809 09:33:27.539320 20982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu as pid 3405
I20260809 09:33:27.539466 20982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786267941944493-20982-0/minicluster-data/ts-1/wal/instance
I20260809 09:33:27.539986 20982 external_mini_cluster.cc:1428] Setting key b3a0e32bda130248f182f3dd2be5f4ce
I20260809 09:33:27.542325 20982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
/tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786267941944493-20982-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786267941944493-20982-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786267941944493-20982-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786267941944493-20982-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.125.131:0
--local_ip_for_outbound_sockets=127.20.125.131
--webserver_interface=127.20.125.131
--webserver_port=0
--tserver_master_addrs=127.20.125.190:39021
--builtin_ntp_servers=127.20.125.148:35005
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260809 09:33:27.543519 3163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:33:27.538994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.130:53041 (local address 127.20.125.190:39021)
0809 09:33:27.539138 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0809 09:33:27.539145 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 09:33:27.540254 (+ 1109us) server_negotiation.cc:407] Connection header received
0809 09:33:27.541766 (+ 1512us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:33:27.541769 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:33:27.541813 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:33:27.541901 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:33:27.543016 (+ 1115us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:33:27.543022 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:33:27.543025 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 09:33:27.543040 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:33:27.543066 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 09:33:27.543080 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:33:27.543192 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0809 09:33:27.543278 (+ 86us) server_negotiation.cc:299] Negotiation successful
0809 09:33:27.543324 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260809 09:33:27.544339 3596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:33:27.539327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:39021 (local address 127.20.125.130:53041)
0809 09:33:27.540078 (+ 751us) negotiation.cc:107] Waiting for socket to connect
0809 09:33:27.540129 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0809 09:33:27.541544 (+ 1415us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:33:27.542019 (+ 475us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:33:27.542026 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:33:27.542109 (+ 83us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:33:27.542758 (+ 649us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 09:33:27.542759 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:33:27.542799 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 09:33:27.542868 (+ 69us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 09:33:27.543165 (+ 297us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 09:33:27.543169 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 09:33:27.543172 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 09:33:27.543290 (+ 118us) client_negotiation.cc:240] Negotiation successful
0809 09:33:27.543466 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":311,"thread_start_us":171,"threads_started":1}
I20260809 09:33:27.545589 3591 heartbeater.cc:347] Connected to a master server at 127.20.125.190:39021
I20260809 09:33:27.545893 3591 heartbeater.cc:464] Registering TS with master...
I20260809 09:33:27.546303 3591 heartbeater.cc:511] Master 127.20.125.190:39021 requested a full tablet report, sending...
I20260809 09:33:27.547422 3103 ts_manager.cc:194] Registered new tserver with Master: 2cdaef2ee29b420abfcf700b002d7716 (127.20.125.130:39041)
W20260809 09:33:27.661767 3599 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 09:33:27.662227 3599 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 09:33:27.662302 3599 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 09:33:27.666760 3599 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 09:33:27.666846 3599 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 09:33:27.666944 3599 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.125.131
I20260809 09:33:27.672920 3599 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.125.148:35005
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786267941944493-20982-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786267941944493-20982-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.125.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786267941944493-20982-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.125.131
--webserver_port=0
--tserver_master_addrs=127.20.125.190:39021
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3599
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.125.131
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786267941944493-20982-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 08:38:55 UTC on ccycloud.quasar-njjleu.root.comops.site
build id 4575935
I20260809 09:33:27.674127 3599 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 09:33:27.675729 3599 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 09:33:27.685649 3605 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:33:27.685789 3599 server_base.cc:1034] running on GCE node
W20260809 09:33:27.685755 3606 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:33:27.685649 3608 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:33:27.686352 3599 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 09:33:27.687055 3599 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 09:33:27.688231 3599 hybrid_clock.cc:648] HybridClock initialized: now 1786268007688216 us; error 26 us; skew 500 ppm
I20260809 09:33:27.694589 3599 webserver.cc:460] Webserver started at http://127.20.125.131:46363/ using document root <none> and password file <none>
I20260809 09:33:27.695268 3599 fs_manager.cc:359] Metadata directory not provided
I20260809 09:33:27.695343 3599 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 09:33:27.695569 3599 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 09:33:27.697726 3599 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786267941944493-20982-0/minicluster-data/ts-2/data/instance:
uuid: "d2e80e7fcd914245a92fb5631f4af895"
format_stamp: "Formatted at 2026-08-09 09:33:27 on dist-test-slave-7kzw"
server_key: "13b5a535b7d49df352db1710e0923a44"
server_key_iv: "018ffcafd7f0944f2f670e09b5bad9e9"
server_key_version: "encryptionkey@0"
I20260809 09:33:27.698328 3599 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786267941944493-20982-0/minicluster-data/ts-2/wal/instance:
uuid: "d2e80e7fcd914245a92fb5631f4af895"
format_stamp: "Formatted at 2026-08-09 09:33:27 on dist-test-slave-7kzw"
server_key: "13b5a535b7d49df352db1710e0923a44"
server_key_iv: "018ffcafd7f0944f2f670e09b5bad9e9"
server_key_version: "encryptionkey@0"
I20260809 09:33:27.702261 3599 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260809 09:33:27.705360 3618 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 09:33:27.706858 3599 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 09:33:27.707019 3599 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786267941944493-20982-0/minicluster-data/ts-2/data,/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786267941944493-20982-0/minicluster-data/ts-2/wal
uuid: "d2e80e7fcd914245a92fb5631f4af895"
format_stamp: "Formatted at 2026-08-09 09:33:27 on dist-test-slave-7kzw"
server_key: "13b5a535b7d49df352db1710e0923a44"
server_key_iv: "018ffcafd7f0944f2f670e09b5bad9e9"
server_key_version: "encryptionkey@0"
I20260809 09:33:27.707152 3599 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786267941944493-20982-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786267941944493-20982-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786267941944493-20982-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 09:33:27.712200 3599 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 09:33:27.712836 3599 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 09:33:27.713035 3599 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 09:33:27.713614 3599 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 09:33:27.714624 3599 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 09:33:27.714696 3599 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 09:33:27.714766 3599 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 09:33:27.714819 3599 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 09:33:27.724150 3599 rpc_server.cc:307] RPC server started. Bound to: 127.20.125.131:44055
I20260809 09:33:27.724318 3785 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.125.131:44055 every 8 connection(s)
I20260809 09:33:27.725170 3599 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786267941944493-20982-0/minicluster-data/ts-2/data/info.pb
I20260809 09:33:27.731777 20982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu as pid 3599
I20260809 09:33:27.731921 20982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786267941944493-20982-0/minicluster-data/ts-2/wal/instance
I20260809 09:33:27.732352 20982 external_mini_cluster.cc:1428] Setting key 399f8f1f9dfeb7d978f13d3acab8106e
I20260809 09:33:27.732396 3163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:33:27.727332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.131:39907 (local address 127.20.125.190:39021)
0809 09:33:27.727499 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0809 09:33:27.727503 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 09:33:27.728636 (+ 1133us) server_negotiation.cc:407] Connection header received
0809 09:33:27.730131 (+ 1495us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:33:27.730134 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:33:27.730175 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:33:27.730257 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:33:27.731746 (+ 1489us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:33:27.731753 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:33:27.731756 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 09:33:27.731774 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:33:27.731805 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 09:33:27.731819 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:33:27.731951 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0809 09:33:27.732172 (+ 221us) server_negotiation.cc:299] Negotiation successful
0809 09:33:27.732233 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260809 09:33:27.733395 3789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:33:27.727741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:39021 (local address 127.20.125.131:39907)
0809 09:33:27.728455 (+ 714us) negotiation.cc:107] Waiting for socket to connect
0809 09:33:27.728501 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 09:33:27.729878 (+ 1377us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:33:27.730450 (+ 572us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:33:27.730458 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:33:27.730630 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:33:27.731373 (+ 743us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 09:33:27.731375 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:33:27.731421 (+ 46us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 09:33:27.731502 (+ 81us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 09:33:27.731972 (+ 470us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 09:33:27.731982 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 09:33:27.731989 (+ 7us) client_negotiation.cc:769] Sending connection context
0809 09:33:27.732169 (+ 180us) client_negotiation.cc:240] Negotiation successful
0809 09:33:27.732372 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":283,"thread_start_us":107,"threads_started":1}
I20260809 09:33:27.734692 3786 heartbeater.cc:347] Connected to a master server at 127.20.125.190:39021
I20260809 09:33:27.734990 3786 heartbeater.cc:464] Registering TS with master...
I20260809 09:33:27.735509 3786 heartbeater.cc:511] Master 127.20.125.190:39021 requested a full tablet report, sending...
I20260809 09:33:27.736734 3103 ts_manager.cc:194] Registered new tserver with Master: d2e80e7fcd914245a92fb5631f4af895 (127.20.125.131:44055)
I20260809 09:33:27.746855 20982 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260809 09:33:27.749769 3163 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 09:33:27.748948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35442 (local address 127.20.125.190:39021)
0809 09:33:27.749074 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0809 09:33:27.749078 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 09:33:27.749258 (+ 180us) server_negotiation.cc:407] Connection header received
0809 09:33:27.749355 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:33:27.749357 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:33:27.749388 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:33:27.749459 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:33:27.749658 (+ 199us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35442: BlockingRecv error: recv got EOF from 127.0.0.1:35442 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
I20260809 09:33:27.749990 20982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu with pid 3197
I20260809 09:33:27.766088 20982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu with pid 3405
I20260809 09:33:27.782708 20982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu with pid 3599
I20260809 09:33:27.793592 20982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu with pid 3065
2026-08-09T09:33:27Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (881 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786267941944493-20982-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 09:33:27 dist-test-slave-7kzw krb5kdc[3801](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 09:33:27 dist-test-slave-7kzw krb5kdc[3801](info): set up 2 sockets
Aug 09 09:33:27 dist-test-slave-7kzw krb5kdc[3801](info): commencing operation
krb5kdc: starting...
W20260809 09:33:29.090926 20982 mini_kdc.cc:121] Time spent starting KDC: real 1.271s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786267941944493-20982-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786267941944493-20982-0/krb5kdc/test-user.keytab.
Aug 09 09:33:29 dist-test-slave-7kzw krb5kdc[3801](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268009, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T09:33:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T09:33:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.125.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.125.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.125.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.190.keytab.
Entry for principal kudu/127.20.125.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.190.keytab.
I20260809 09:33:29.229836 20982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
/tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786267941944493-20982-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786267941944493-20982-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786267941944493-20982-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786267941944493-20982-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.125.190:33643
--webserver_interface=127.20.125.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.190.keytab
--principal=kudu/127.20.125.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.125.148:37311
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.125.190:33643
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786267941944493-20982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786267941944493-20982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786267941944493-20982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 09:33:29.345425 3820 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 09:33:29.345708 3820 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 09:33:29.345750 3820 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 09:33:29.349864 3820 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 09:33:29.349937 3820 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 09:33:29.349957 3820 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 09:33:29.349975 3820 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 09:33:29.349990 3820 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 09:33:29.355515 3820 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.125.148:37311
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786267941944493-20982-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786267941944493-20982-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.125.190:33643
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.190.keytab
--principal=kudu/127.20.125.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.125.190:33643
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786267941944493-20982-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.125.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3820
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786267941944493-20982-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 08:38:55 UTC on ccycloud.quasar-njjleu.root.comops.site
build id 4575935
I20260809 09:33:29.356603 3820 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 09:33:29.357664 3820 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 09:33:29.366238 3829 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:33:29.366317 3826 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:33:29.366266 3825 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:33:29.366456 3820 server_base.cc:1034] running on GCE node
I20260809 09:33:29.367102 3820 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 09:33:29.368207 3820 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 09:33:29.369427 3820 hybrid_clock.cc:648] HybridClock initialized: now 1786268009369352 us; error 101 us; skew 500 ppm
Aug 09 09:33:29 dist-test-slave-7kzw krb5kdc[3801](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268009, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 09:33:29.378285 3820 init.cc:377] Logged in from keytab as kudu/127.20.125.190@KRBTEST.COM (short username kudu)
I20260809 09:33:29.379379 3820 webserver.cc:460] Webserver started at http://127.20.125.190:38103/ using document root <none> and password file <none>
I20260809 09:33:29.379992 3820 fs_manager.cc:359] Metadata directory not provided
I20260809 09:33:29.380074 3820 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 09:33:29.380255 3820 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 09:33:29.382328 3820 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786267941944493-20982-0/minicluster-data/master-0/data/instance:
uuid: "7531f1facac5426986b4c85c1903c3f9"
format_stamp: "Formatted at 2026-08-09 09:33:29 on dist-test-slave-7kzw"
server_key: "1d96b0afecdf19513b9dbf5bfc1ceb05"
server_key_iv: "a951dcd6dfff4e4c569016619b42860e"
server_key_version: "encryptionkey@0"
I20260809 09:33:29.382965 3820 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786267941944493-20982-0/minicluster-data/master-0/wal/instance:
uuid: "7531f1facac5426986b4c85c1903c3f9"
format_stamp: "Formatted at 2026-08-09 09:33:29 on dist-test-slave-7kzw"
server_key: "1d96b0afecdf19513b9dbf5bfc1ceb05"
server_key_iv: "a951dcd6dfff4e4c569016619b42860e"
server_key_version: "encryptionkey@0"
I20260809 09:33:29.386876 3820 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260809 09:33:29.389366 3840 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 09:33:29.390717 3820 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 09:33:29.390844 3820 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786267941944493-20982-0/minicluster-data/master-0/data,/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786267941944493-20982-0/minicluster-data/master-0/wal
uuid: "7531f1facac5426986b4c85c1903c3f9"
format_stamp: "Formatted at 2026-08-09 09:33:29 on dist-test-slave-7kzw"
server_key: "1d96b0afecdf19513b9dbf5bfc1ceb05"
server_key_iv: "a951dcd6dfff4e4c569016619b42860e"
server_key_version: "encryptionkey@0"
I20260809 09:33:29.390938 3820 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786267941944493-20982-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786267941944493-20982-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786267941944493-20982-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 09:33:29.427570 3820 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 09:33:29.433879 3820 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 09:33:29.434186 3820 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 09:33:29.444656 3820 rpc_server.cc:307] RPC server started. Bound to: 127.20.125.190:33643
I20260809 09:33:29.444687 3911 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.125.190:33643 every 8 connection(s)
I20260809 09:33:29.446175 3820 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786267941944493-20982-0/minicluster-data/master-0/data/info.pb
I20260809 09:33:29.448086 20982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu as pid 3820
I20260809 09:33:29.448233 20982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786267941944493-20982-0/minicluster-data/master-0/wal/instance
I20260809 09:33:29.448540 20982 external_mini_cluster.cc:1428] Setting key 37bc9a85c6f5337b11b79571d636c12f
I20260809 09:33:29.450450 3913 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 09:33:29.459569 3913 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7531f1facac5426986b4c85c1903c3f9: Bootstrap starting.
Aug 09 09:33:29 dist-test-slave-7kzw krb5kdc[3801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268009, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.125.190@KRBTEST.COM
I20260809 09:33:29.462967 3913 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7531f1facac5426986b4c85c1903c3f9: Neither blocks nor log segments found. Creating new log.
I20260809 09:33:29.464028 3913 log.cc:824] T 00000000000000000000000000000000 P 7531f1facac5426986b4c85c1903c3f9: Log is configured to *not* fsync() on all Append() calls
I20260809 09:33:29.466637 3913 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7531f1facac5426986b4c85c1903c3f9: No bootstrap required, opened a new log
I20260809 09:33:29.466825 3916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:33:29.450154 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38284 (local address 127.20.125.190:33643)
0809 09:33:29.450782 (+ 628us) server_negotiation.cc:206] Beginning negotiation
0809 09:33:29.450810 (+ 28us) server_negotiation.cc:399] Waiting for connection header
0809 09:33:29.450874 (+ 64us) server_negotiation.cc:407] Connection header received
0809 09:33:29.451950 (+ 1076us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:33:29.451978 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:33:29.452473 (+ 495us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:33:29.452895 (+ 422us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:33:29.453818 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:33:29.455337 (+ 1519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:33:29.456774 (+ 1437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:33:29.457582 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:33:29.457756 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:33:29.460980 (+ 3224us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:33:29.460986 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:33:29.461000 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:33:29.461031 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:33:29.463334 (+ 2303us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:33:29.463826 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:33:29.463833 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:33:29.463840 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:33:29.463976 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:33:29.464294 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:33:29.464299 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:33:29.464301 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:33:29.464858 (+ 557us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:33:29.465107 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0809 09:33:29.465520 (+ 413us) server_negotiation.cc:299] Negotiation successful
0809 09:33:29.465911 (+ 391us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":438,"thread_start_us":198,"threads_started":1}
I20260809 09:33:29.470423 3913 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7531f1facac5426986b4c85c1903c3f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7531f1facac5426986b4c85c1903c3f9" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 33643 } }
I20260809 09:33:29.470788 3913 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7531f1facac5426986b4c85c1903c3f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 09:33:29.470894 3913 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7531f1facac5426986b4c85c1903c3f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7531f1facac5426986b4c85c1903c3f9, State: Initialized, Role: FOLLOWER
I20260809 09:33:29.471354 3913 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7531f1facac5426986b4c85c1903c3f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7531f1facac5426986b4c85c1903c3f9" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 33643 } }
I20260809 09:33:29.471520 3913 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7531f1facac5426986b4c85c1903c3f9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 09:33:29.471626 3913 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7531f1facac5426986b4c85c1903c3f9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 09:33:29.471758 3913 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7531f1facac5426986b4c85c1903c3f9 [term 0 FOLLOWER]: Advancing to term 1
I20260809 09:33:29.472802 3913 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7531f1facac5426986b4c85c1903c3f9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7531f1facac5426986b4c85c1903c3f9" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 33643 } }
I20260809 09:33:29.473176 3913 leader_election.cc:302] T 00000000000000000000000000000000 P 7531f1facac5426986b4c85c1903c3f9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7531f1facac5426986b4c85c1903c3f9; no voters:
I20260809 09:33:29.473552 3913 leader_election.cc:288] T 00000000000000000000000000000000 P 7531f1facac5426986b4c85c1903c3f9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 09:33:29.473680 3919 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7531f1facac5426986b4c85c1903c3f9 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 09:33:29.473948 3919 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7531f1facac5426986b4c85c1903c3f9 [term 1 LEADER]: Becoming Leader. State: Replica: 7531f1facac5426986b4c85c1903c3f9, State: Running, Role: LEADER
I20260809 09:33:29.474277 3919 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7531f1facac5426986b4c85c1903c3f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7531f1facac5426986b4c85c1903c3f9" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 33643 } }
I20260809 09:33:29.474977 3913 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7531f1facac5426986b4c85c1903c3f9 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 09:33:29.476063 3922 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7531f1facac5426986b4c85c1903c3f9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7531f1facac5426986b4c85c1903c3f9. Latest consensus state: current_term: 1 leader_uuid: "7531f1facac5426986b4c85c1903c3f9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7531f1facac5426986b4c85c1903c3f9" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 33643 } } }
I20260809 09:33:29.476274 3922 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7531f1facac5426986b4c85c1903c3f9 [sys.catalog]: This master's current role is: LEADER
I20260809 09:33:29.476646 3931 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 09:33:29.476567 3921 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7531f1facac5426986b4c85c1903c3f9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7531f1facac5426986b4c85c1903c3f9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7531f1facac5426986b4c85c1903c3f9" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 33643 } } }
I20260809 09:33:29.477212 3921 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7531f1facac5426986b4c85c1903c3f9 [sys.catalog]: This master's current role is: LEADER
I20260809 09:33:29.479836 3931 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 09:33:29.485677 3931 catalog_manager.cc:1347] Generated new cluster ID: 083520a0a15540bb9d6477269f446535
I20260809 09:33:29.485764 3931 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 09:33:29.513836 3931 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 09:33:29.515334 3931 catalog_manager.cc:1504] Loading token signing keys...
I20260809 09:33:29.528268 3931 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7531f1facac5426986b4c85c1903c3f9: Generated new TSK 0
I20260809 09:33:29.529223 3931 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.20.125.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.125.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.125.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.129.keytab.
Entry for principal kudu/127.20.125.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.129.keytab.
I20260809 09:33:29.573855 20982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
/tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786267941944493-20982-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786267941944493-20982-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786267941944493-20982-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786267941944493-20982-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.125.129:0
--local_ip_for_outbound_sockets=127.20.125.129
--webserver_interface=127.20.125.129
--webserver_port=0
--tserver_master_addrs=127.20.125.190:33643
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.129.keytab
--principal=kudu/127.20.125.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.125.148:37311
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786267941944493-20982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786267941944493-20982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786267941944493-20982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 09:33:29.692993 3946 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 09:33:29.693301 3946 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 09:33:29.693374 3946 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 09:33:29.697551 3946 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 09:33:29.697647 3946 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 09:33:29.697736 3946 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.125.129
I20260809 09:33:29.703217 3946 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.125.148:37311
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786267941944493-20982-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786267941944493-20982-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.129.keytab
--principal=kudu/127.20.125.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.125.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786267941944493-20982-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.125.129
--webserver_port=0
--tserver_master_addrs=127.20.125.190:33643
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3946
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.125.129
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786267941944493-20982-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 08:38:55 UTC on ccycloud.quasar-njjleu.root.comops.site
build id 4575935
I20260809 09:33:29.704312 3946 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 09:33:29.705363 3946 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 09:33:29.715157 3957 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:33:29.715153 3955 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:33:29.715153 3954 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:33:29.715456 3946 server_base.cc:1034] running on GCE node
I20260809 09:33:29.715991 3946 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 09:33:29.716660 3946 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 09:33:29.717864 3946 hybrid_clock.cc:648] HybridClock initialized: now 1786268009717829 us; error 47 us; skew 500 ppm
Aug 09 09:33:29 dist-test-slave-7kzw krb5kdc[3801](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268009, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 09:33:29.720836 3946 init.cc:377] Logged in from keytab as kudu/127.20.125.129@KRBTEST.COM (short username kudu)
I20260809 09:33:29.721719 3946 webserver.cc:460] Webserver started at http://127.20.125.129:43995/ using document root <none> and password file <none>
I20260809 09:33:29.722256 3946 fs_manager.cc:359] Metadata directory not provided
I20260809 09:33:29.722317 3946 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 09:33:29.722553 3946 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 09:33:29.724572 3946 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786267941944493-20982-0/minicluster-data/ts-0/data/instance:
uuid: "b7bd860b388e4818b53dae80c142be08"
format_stamp: "Formatted at 2026-08-09 09:33:29 on dist-test-slave-7kzw"
server_key: "517c5e4f4d0ad950d45b619e5ea952da"
server_key_iv: "9e18ba5bc89eecd742ec1e8c68a35755"
server_key_version: "encryptionkey@0"
I20260809 09:33:29.725113 3946 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786267941944493-20982-0/minicluster-data/ts-0/wal/instance:
uuid: "b7bd860b388e4818b53dae80c142be08"
format_stamp: "Formatted at 2026-08-09 09:33:29 on dist-test-slave-7kzw"
server_key: "517c5e4f4d0ad950d45b619e5ea952da"
server_key_iv: "9e18ba5bc89eecd742ec1e8c68a35755"
server_key_version: "encryptionkey@0"
I20260809 09:33:29.728927 3946 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.001s
I20260809 09:33:29.731537 3969 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 09:33:29.732872 3946 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 09:33:29.733008 3946 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786267941944493-20982-0/minicluster-data/ts-0/data,/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786267941944493-20982-0/minicluster-data/ts-0/wal
uuid: "b7bd860b388e4818b53dae80c142be08"
format_stamp: "Formatted at 2026-08-09 09:33:29 on dist-test-slave-7kzw"
server_key: "517c5e4f4d0ad950d45b619e5ea952da"
server_key_iv: "9e18ba5bc89eecd742ec1e8c68a35755"
server_key_version: "encryptionkey@0"
I20260809 09:33:29.733125 3946 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786267941944493-20982-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786267941944493-20982-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786267941944493-20982-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 09:33:29.749303 3946 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 09:33:29.752733 3946 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 09:33:29.752944 3946 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 09:33:29.753585 3946 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 09:33:29.754521 3946 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 09:33:29.754585 3946 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 09:33:29.754650 3946 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 09:33:29.754680 3946 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 09:33:29.764433 3946 rpc_server.cc:307] RPC server started. Bound to: 127.20.125.129:34895
I20260809 09:33:29.764456 4132 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.125.129:34895 every 8 connection(s)
I20260809 09:33:29.765395 3946 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786267941944493-20982-0/minicluster-data/ts-0/data/info.pb
I20260809 09:33:29.772298 20982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu as pid 3946
I20260809 09:33:29.772398 20982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786267941944493-20982-0/minicluster-data/ts-0/wal/instance
I20260809 09:33:29.772698 20982 external_mini_cluster.cc:1428] Setting key 7b5674656720f37afe714bb4748378f0
Aug 09 09:33:29 dist-test-slave-7kzw krb5kdc[3801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268009, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.129@KRBTEST.COM for kudu/127.20.125.190@KRBTEST.COM
I20260809 09:33:29.780635 3916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:33:29.767384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.129:41987 (local address 127.20.125.190:33643)
0809 09:33:29.767523 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0809 09:33:29.767527 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 09:33:29.768352 (+ 825us) server_negotiation.cc:407] Connection header received
0809 09:33:29.769813 (+ 1461us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:33:29.769816 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:33:29.769878 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:33:29.769964 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:33:29.771229 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:33:29.771782 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:33:29.773055 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:33:29.773325 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:33:29.773448 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:33:29.776194 (+ 2746us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:33:29.776200 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:33:29.776208 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:33:29.776233 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:33:29.777950 (+ 1717us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:33:29.778572 (+ 622us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:33:29.778579 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:33:29.778584 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:33:29.778688 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:33:29.779051 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:33:29.779058 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:33:29.779062 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:33:29.779386 (+ 324us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:33:29.779496 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0809 09:33:29.780328 (+ 832us) server_negotiation.cc:299] Negotiation successful
0809 09:33:29.780502 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260809 09:33:29.781877 4135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:33:29.767714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:33643 (local address 127.20.125.129:41987)
0809 09:33:29.768195 (+ 481us) negotiation.cc:107] Waiting for socket to connect
0809 09:33:29.768241 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 09:33:29.769587 (+ 1346us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:33:29.770133 (+ 546us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:33:29.770141 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:33:29.770577 (+ 436us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:33:29.771076 (+ 499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:33:29.771089 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:33:29.771919 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:33:29.771927 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:33:29.772924 (+ 997us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:33:29.772935 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:33:29.773490 (+ 555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:33:29.773494 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:33:29.773673 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:33:29.774312 (+ 639us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 09:33:29.774329 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:33:29.776023 (+ 1694us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 09:33:29.778092 (+ 2069us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:33:29.778097 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:33:29.778110 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:33:29.778401 (+ 291us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:33:29.778794 (+ 393us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:33:29.778796 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:33:29.778798 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:33:29.778941 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:33:29.779560 (+ 619us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 09:33:29.779571 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 09:33:29.779974 (+ 403us) client_negotiation.cc:769] Sending connection context
0809 09:33:29.780209 (+ 235us) client_negotiation.cc:240] Negotiation successful
0809 09:33:29.780618 (+ 409us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":289,"thread_start_us":119,"threads_started":1}
I20260809 09:33:29.783464 4133 heartbeater.cc:347] Connected to a master server at 127.20.125.190:33643
I20260809 09:33:29.783772 4133 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.20.125.130@KRBTEST.COM; defaulting to no policy
I20260809 09:33:29.784438 4133 heartbeater.cc:511] Master 127.20.125.190:33643 requested a full tablet report, sending...
I20260809 09:33:29.786197 3864 ts_manager.cc:194] Registered new tserver with Master: b7bd860b388e4818b53dae80c142be08 (127.20.125.129:34895)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.125.130@KRBTEST.COM" created.
I20260809 09:33:29.787664 3864 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.125.129@KRBTEST.COM'} at 127.20.125.129:41987
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.125.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.130.keytab.
Entry for principal kudu/127.20.125.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.130.keytab.
I20260809 09:33:29.806458 20982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
/tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786267941944493-20982-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786267941944493-20982-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786267941944493-20982-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786267941944493-20982-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.125.130:0
--local_ip_for_outbound_sockets=127.20.125.130
--webserver_interface=127.20.125.130
--webserver_port=0
--tserver_master_addrs=127.20.125.190:33643
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.130.keytab
--principal=kudu/127.20.125.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.125.148:37311
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786267941944493-20982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786267941944493-20982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786267941944493-20982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 09:33:29.929500 4141 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 09:33:29.929774 4141 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 09:33:29.929818 4141 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 09:33:29.933943 4141 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 09:33:29.934001 4141 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 09:33:29.934059 4141 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.125.130
I20260809 09:33:29.939841 4141 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.125.148:37311
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786267941944493-20982-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786267941944493-20982-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.130.keytab
--principal=kudu/127.20.125.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.125.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786267941944493-20982-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.125.130
--webserver_port=0
--tserver_master_addrs=127.20.125.190:33643
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4141
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.125.130
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786267941944493-20982-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 08:38:55 UTC on ccycloud.quasar-njjleu.root.comops.site
build id 4575935
I20260809 09:33:29.941402 4141 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 09:33:29.942513 4141 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 09:33:29.952823 4146 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:33:29.952826 4152 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:33:29.952823 4148 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:33:29.953121 4141 server_base.cc:1034] running on GCE node
I20260809 09:33:29.953688 4141 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 09:33:29.954566 4141 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 09:33:29.955778 4141 hybrid_clock.cc:648] HybridClock initialized: now 1786268009955729 us; error 51 us; skew 500 ppm
Aug 09 09:33:29 dist-test-slave-7kzw krb5kdc[3801](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268009, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 09:33:29.959048 4141 init.cc:377] Logged in from keytab as kudu/127.20.125.130@KRBTEST.COM (short username kudu)
I20260809 09:33:29.960034 4141 webserver.cc:460] Webserver started at http://127.20.125.130:42549/ using document root <none> and password file <none>
I20260809 09:33:29.960616 4141 fs_manager.cc:359] Metadata directory not provided
I20260809 09:33:29.960683 4141 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 09:33:29.960863 4141 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 09:33:29.963045 4141 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786267941944493-20982-0/minicluster-data/ts-1/data/instance:
uuid: "3f80f6138ee847598719f77e05a60190"
format_stamp: "Formatted at 2026-08-09 09:33:29 on dist-test-slave-7kzw"
server_key: "6e17561fc59a81d6f84c77ab9f354717"
server_key_iv: "a0ca07c79172574417084bcb6b11d0e9"
server_key_version: "encryptionkey@0"
I20260809 09:33:29.963701 4141 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786267941944493-20982-0/minicluster-data/ts-1/wal/instance:
uuid: "3f80f6138ee847598719f77e05a60190"
format_stamp: "Formatted at 2026-08-09 09:33:29 on dist-test-slave-7kzw"
server_key: "6e17561fc59a81d6f84c77ab9f354717"
server_key_iv: "a0ca07c79172574417084bcb6b11d0e9"
server_key_version: "encryptionkey@0"
I20260809 09:33:29.968003 4141 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260809 09:33:29.970748 4159 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 09:33:29.972131 4141 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 09:33:29.972270 4141 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786267941944493-20982-0/minicluster-data/ts-1/data,/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786267941944493-20982-0/minicluster-data/ts-1/wal
uuid: "3f80f6138ee847598719f77e05a60190"
format_stamp: "Formatted at 2026-08-09 09:33:29 on dist-test-slave-7kzw"
server_key: "6e17561fc59a81d6f84c77ab9f354717"
server_key_iv: "a0ca07c79172574417084bcb6b11d0e9"
server_key_version: "encryptionkey@0"
I20260809 09:33:29.972391 4141 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786267941944493-20982-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786267941944493-20982-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786267941944493-20982-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 09:33:29.993755 4141 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 09:33:29.997129 4141 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 09:33:29.997366 4141 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 09:33:29.998042 4141 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 09:33:29.999045 4141 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 09:33:29.999100 4141 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 09:33:29.999178 4141 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 09:33:29.999228 4141 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 09:33:30.009627 4141 rpc_server.cc:307] RPC server started. Bound to: 127.20.125.130:33091
I20260809 09:33:30.009657 4315 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.125.130:33091 every 8 connection(s)
I20260809 09:33:30.010720 4141 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786267941944493-20982-0/minicluster-data/ts-1/data/info.pb
I20260809 09:33:30.015123 20982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu as pid 4141
I20260809 09:33:30.015278 20982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786267941944493-20982-0/minicluster-data/ts-1/wal/instance
I20260809 09:33:30.015640 20982 external_mini_cluster.cc:1428] Setting key 443d7c35efb0abfcd2665d81b51f6d3d
Aug 09 09:33:30 dist-test-slave-7kzw krb5kdc[3801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268009, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.130@KRBTEST.COM for kudu/127.20.125.190@KRBTEST.COM
I20260809 09:33:30.028271 3916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:33:30.012747 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.130:49899 (local address 127.20.125.190:33643)
0809 09:33:30.012902 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0809 09:33:30.012906 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 09:33:30.013741 (+ 835us) server_negotiation.cc:407] Connection header received
0809 09:33:30.015591 (+ 1850us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:33:30.015594 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:33:30.015645 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:33:30.015764 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:33:30.017227 (+ 1463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:33:30.017868 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:33:30.019875 (+ 2007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:33:30.020299 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:33:30.020440 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:33:30.023365 (+ 2925us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:33:30.023371 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:33:30.023377 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:33:30.023411 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:33:30.025527 (+ 2116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:33:30.026098 (+ 571us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:33:30.026103 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:33:30.026105 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:33:30.026219 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:33:30.026647 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:33:30.026651 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:33:30.026654 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:33:30.026939 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:33:30.027070 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0809 09:33:30.027818 (+ 748us) server_negotiation.cc:299] Negotiation successful
0809 09:33:30.028012 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
WARNING: no policy specified for kudu/127.20.125.131@KRBTEST.COM; defaulting to no policy
I20260809 09:33:30.029202 4319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:33:30.013110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:33643 (local address 127.20.125.130:49899)
0809 09:33:30.013576 (+ 466us) negotiation.cc:107] Waiting for socket to connect
0809 09:33:30.013622 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 09:33:30.015339 (+ 1717us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:33:30.015962 (+ 623us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:33:30.015972 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:33:30.016480 (+ 508us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:33:30.017020 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:33:30.017061 (+ 41us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:33:30.018027 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:33:30.018034 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:33:30.019694 (+ 1660us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:33:30.019710 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:33:30.020467 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:33:30.020470 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:33:30.020617 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:33:30.021386 (+ 769us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 09:33:30.021404 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:33:30.023161 (+ 1757us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 09:33:30.025663 (+ 2502us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:33:30.025668 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:33:30.025681 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:33:30.025971 (+ 290us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:33:30.026326 (+ 355us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:33:30.026328 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:33:30.026330 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:33:30.026514 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:33:30.027099 (+ 585us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 09:33:30.027105 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 09:33:30.027399 (+ 294us) client_negotiation.cc:769] Sending connection context
0809 09:33:30.027696 (+ 297us) client_negotiation.cc:240] Negotiation successful
0809 09:33:30.028080 (+ 384us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":270,"thread_start_us":106,"threads_started":1}
I20260809 09:33:30.030751 4316 heartbeater.cc:347] Connected to a master server at 127.20.125.190:33643
I20260809 09:33:30.031062 4316 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.125.131@KRBTEST.COM" created.
I20260809 09:33:30.031737 4316 heartbeater.cc:511] Master 127.20.125.190:33643 requested a full tablet report, sending...
I20260809 09:33:30.032977 3864 ts_manager.cc:194] Registered new tserver with Master: 3f80f6138ee847598719f77e05a60190 (127.20.125.130:33091)
I20260809 09:33:30.033906 3864 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.125.130@KRBTEST.COM'} at 127.20.125.130:49899
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.125.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.131.keytab.
Entry for principal kudu/127.20.125.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.131.keytab.
I20260809 09:33:30.050789 20982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
/tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786267941944493-20982-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786267941944493-20982-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786267941944493-20982-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786267941944493-20982-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.125.131:0
--local_ip_for_outbound_sockets=127.20.125.131
--webserver_interface=127.20.125.131
--webserver_port=0
--tserver_master_addrs=127.20.125.190:33643
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.131.keytab
--principal=kudu/127.20.125.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.125.148:37311
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786267941944493-20982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786267941944493-20982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786267941944493-20982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 09:33:30.175294 4323 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 09:33:30.175625 4323 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 09:33:30.175705 4323 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 09:33:30.180526 4323 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 09:33:30.180624 4323 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 09:33:30.180721 4323 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.125.131
I20260809 09:33:30.186352 4323 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.125.148:37311
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786267941944493-20982-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786267941944493-20982-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.131.keytab
--principal=kudu/127.20.125.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.125.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786267941944493-20982-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.125.131
--webserver_port=0
--tserver_master_addrs=127.20.125.190:33643
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4323
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.125.131
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786267941944493-20982-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 08:38:55 UTC on ccycloud.quasar-njjleu.root.comops.site
build id 4575935
I20260809 09:33:30.187589 4323 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 09:33:30.188694 4323 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 09:33:30.199167 4331 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:33:30.199108 4332 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:33:30.199170 4336 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:33:30.199615 4323 server_base.cc:1034] running on GCE node
I20260809 09:33:30.200122 4323 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 09:33:30.200809 4323 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 09:33:30.202047 4323 hybrid_clock.cc:648] HybridClock initialized: now 1786268010202025 us; error 30 us; skew 500 ppm
Aug 09 09:33:30 dist-test-slave-7kzw krb5kdc[3801](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268010, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 09:33:30.205106 4323 init.cc:377] Logged in from keytab as kudu/127.20.125.131@KRBTEST.COM (short username kudu)
I20260809 09:33:30.205983 4323 webserver.cc:460] Webserver started at http://127.20.125.131:43063/ using document root <none> and password file <none>
I20260809 09:33:30.206617 4323 fs_manager.cc:359] Metadata directory not provided
I20260809 09:33:30.206692 4323 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 09:33:30.206875 4323 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 09:33:30.209064 4323 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786267941944493-20982-0/minicluster-data/ts-2/data/instance:
uuid: "f822542f73ca42d6aad42efbc11efdc5"
format_stamp: "Formatted at 2026-08-09 09:33:30 on dist-test-slave-7kzw"
server_key: "0af81bf0af34222750dc1be8f075475a"
server_key_iv: "df395bdd73772cd5bbf0b0e5b52f07a6"
server_key_version: "encryptionkey@0"
I20260809 09:33:30.209616 4323 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786267941944493-20982-0/minicluster-data/ts-2/wal/instance:
uuid: "f822542f73ca42d6aad42efbc11efdc5"
format_stamp: "Formatted at 2026-08-09 09:33:30 on dist-test-slave-7kzw"
server_key: "0af81bf0af34222750dc1be8f075475a"
server_key_iv: "df395bdd73772cd5bbf0b0e5b52f07a6"
server_key_version: "encryptionkey@0"
I20260809 09:33:30.213886 4323 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.004s
I20260809 09:33:30.216601 4346 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 09:33:30.218051 4323 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 09:33:30.218231 4323 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786267941944493-20982-0/minicluster-data/ts-2/data,/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786267941944493-20982-0/minicluster-data/ts-2/wal
uuid: "f822542f73ca42d6aad42efbc11efdc5"
format_stamp: "Formatted at 2026-08-09 09:33:30 on dist-test-slave-7kzw"
server_key: "0af81bf0af34222750dc1be8f075475a"
server_key_iv: "df395bdd73772cd5bbf0b0e5b52f07a6"
server_key_version: "encryptionkey@0"
I20260809 09:33:30.218390 4323 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786267941944493-20982-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786267941944493-20982-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786267941944493-20982-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 09:33:30.229787 4323 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 09:33:30.233278 4323 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 09:33:30.233449 4323 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 09:33:30.234040 4323 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 09:33:30.234899 4323 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 09:33:30.234951 4323 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 09:33:30.234993 4323 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 09:33:30.235008 4323 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 09:33:30.244354 4323 rpc_server.cc:307] RPC server started. Bound to: 127.20.125.131:32931
I20260809 09:33:30.244371 4517 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.125.131:32931 every 8 connection(s)
I20260809 09:33:30.245306 4323 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786267941944493-20982-0/minicluster-data/ts-2/data/info.pb
I20260809 09:33:30.249063 20982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu as pid 4323
I20260809 09:33:30.249254 20982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786267941944493-20982-0/minicluster-data/ts-2/wal/instance
I20260809 09:33:30.249580 20982 external_mini_cluster.cc:1428] Setting key 20d231da851e080d7af631c2da5f6d70
Aug 09 09:33:30 dist-test-slave-7kzw krb5kdc[3801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268010, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.131@KRBTEST.COM for kudu/127.20.125.190@KRBTEST.COM
I20260809 09:33:30.260512 3916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:33:30.247244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.131:47157 (local address 127.20.125.190:33643)
0809 09:33:30.247396 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0809 09:33:30.247400 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 09:33:30.248322 (+ 922us) server_negotiation.cc:407] Connection header received
0809 09:33:30.249872 (+ 1550us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:33:30.249876 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:33:30.249935 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:33:30.250037 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:33:30.251736 (+ 1699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:33:30.252416 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:33:30.253644 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:33:30.253892 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:33:30.253991 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:33:30.256720 (+ 2729us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:33:30.256723 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:33:30.256725 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:33:30.256752 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:33:30.258222 (+ 1470us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:33:30.258849 (+ 627us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:33:30.258852 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:33:30.258853 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:33:30.258943 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:33:30.259388 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:33:30.259392 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:33:30.259393 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:33:30.259592 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:33:30.259681 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0809 09:33:30.260270 (+ 589us) server_negotiation.cc:299] Negotiation successful
0809 09:33:30.260381 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260809 09:33:30.261368 4520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:33:30.247633 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:33643 (local address 127.20.125.131:47157)
0809 09:33:30.248152 (+ 519us) negotiation.cc:107] Waiting for socket to connect
0809 09:33:30.248196 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 09:33:30.249619 (+ 1423us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:33:30.250341 (+ 722us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:33:30.250349 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:33:30.250946 (+ 597us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:33:30.251585 (+ 639us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:33:30.251601 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:33:30.252549 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:33:30.252552 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:33:30.253448 (+ 896us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:33:30.253458 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:33:30.254054 (+ 596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:33:30.254057 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:33:30.254233 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:33:30.254922 (+ 689us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 09:33:30.254939 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:33:30.256532 (+ 1593us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 09:33:30.258367 (+ 1835us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:33:30.258372 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:33:30.258394 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:33:30.258748 (+ 354us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:33:30.259065 (+ 317us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:33:30.259068 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:33:30.259070 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:33:30.259236 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:33:30.259764 (+ 528us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 09:33:30.259769 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 09:33:30.260025 (+ 256us) client_negotiation.cc:769] Sending connection context
0809 09:33:30.260196 (+ 171us) client_negotiation.cc:240] Negotiation successful
0809 09:33:30.260477 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":284,"thread_start_us":106,"threads_started":1}
I20260809 09:33:30.262705 4518 heartbeater.cc:347] Connected to a master server at 127.20.125.190:33643
I20260809 09:33:30.263000 4518 heartbeater.cc:464] Registering TS with master...
I20260809 09:33:30.263679 4518 heartbeater.cc:511] Master 127.20.125.190:33643 requested a full tablet report, sending...
I20260809 09:33:30.264773 3864 ts_manager.cc:194] Registered new tserver with Master: f822542f73ca42d6aad42efbc11efdc5 (127.20.125.131:32931)
I20260809 09:33:30.265357 3864 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.125.131@KRBTEST.COM'} at 127.20.125.131:47157
I20260809 09:33:30.275966 20982 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 09:33:30.286192 3916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:33:30.278194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38300 (local address 127.20.125.190:33643)
0809 09:33:30.278327 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0809 09:33:30.278332 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 09:33:30.278535 (+ 203us) server_negotiation.cc:407] Connection header received
0809 09:33:30.278701 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:33:30.278710 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:33:30.278752 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:33:30.278834 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:33:30.279559 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:33:30.280072 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:33:30.281077 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:33:30.281342 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:33:30.281447 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:33:30.282926 (+ 1479us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:33:30.282930 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:33:30.282933 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:33:30.282958 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:33:30.284382 (+ 1424us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:33:30.284843 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:33:30.284848 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:33:30.284850 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:33:30.284940 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:33:30.285260 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:33:30.285264 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:33:30.285266 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:33:30.285496 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:33:30.285650 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0809 09:33:30.285890 (+ 240us) server_negotiation.cc:299] Negotiation successful
0809 09:33:30.286011 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260809 09:33:30.289215 3864 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38300:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 09:33:30.292369 3864 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 09:33:30.307154 4533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:33:30.301391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.130:33091 (local address 127.0.0.1:40256)
0809 09:33:30.302494 (+ 1103us) negotiation.cc:107] Waiting for socket to connect
0809 09:33:30.302509 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 09:33:30.302673 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:33:30.303215 (+ 542us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:33:30.303221 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:33:30.303243 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 09:33:30.303381 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:33:30.303387 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:33:30.304511 (+ 1124us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:33:30.304514 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:33:30.305718 (+ 1204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:33:30.305728 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:33:30.306604 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:33:30.306608 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:33:30.306717 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:33:30.306759 (+ 42us) client_negotiation.cc:769] Sending connection context
0809 09:33:30.306880 (+ 121us) client_negotiation.cc:240] Negotiation successful
0809 09:33:30.306995 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1015,"thread_start_us":115,"threads_started":1}
I20260809 09:33:30.307180 4536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:33:30.301633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40256 (local address 127.20.125.130:33091)
0809 09:33:30.302070 (+ 437us) server_negotiation.cc:206] Beginning negotiation
0809 09:33:30.302083 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 09:33:30.302597 (+ 514us) server_negotiation.cc:407] Connection header received
0809 09:33:30.302843 (+ 246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:33:30.302849 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:33:30.303045 (+ 196us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:33:30.303216 (+ 171us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 09:33:30.303508 (+ 292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:33:30.304371 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:33:30.305842 (+ 1471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:33:30.306440 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:33:30.306630 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:33:30.306774 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0809 09:33:30.306916 (+ 142us) server_negotiation.cc:299] Negotiation successful
0809 09:33:30.307014 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":336,"thread_start_us":129,"threads_started":1}
I20260809 09:33:30.307968 4538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:33:30.302177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.129:34895 (local address 127.0.0.1:48446)
0809 09:33:30.302904 (+ 727us) negotiation.cc:107] Waiting for socket to connect
0809 09:33:30.302917 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 09:33:30.302995 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:33:30.303562 (+ 567us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:33:30.303566 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:33:30.303584 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 09:33:30.303673 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:33:30.303679 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:33:30.305004 (+ 1325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:33:30.305007 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:33:30.306304 (+ 1297us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:33:30.306316 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:33:30.307515 (+ 1199us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:33:30.307518 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:33:30.307623 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:33:30.307645 (+ 22us) client_negotiation.cc:769] Sending connection context
0809 09:33:30.307753 (+ 108us) client_negotiation.cc:240] Negotiation successful
0809 09:33:30.307825 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":672,"mutex_wait_us":55,"thread_start_us":110,"threads_started":1}
I20260809 09:33:30.307998 4539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:33:30.302662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48446 (local address 127.20.125.129:34895)
0809 09:33:30.303160 (+ 498us) server_negotiation.cc:206] Beginning negotiation
0809 09:33:30.303167 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 09:33:30.303189 (+ 22us) server_negotiation.cc:407] Connection header received
0809 09:33:30.303268 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:33:30.303272 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:33:30.303445 (+ 173us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:33:30.303593 (+ 148us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 09:33:30.303772 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:33:30.304889 (+ 1117us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:33:30.306505 (+ 1616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:33:30.307322 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:33:30.307452 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:33:30.307582 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0809 09:33:30.307785 (+ 203us) server_negotiation.cc:299] Negotiation successful
0809 09:33:30.307870 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":373,"thread_start_us":120,"threads_started":1}
I20260809 09:33:30.308573 4532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:33:30.301342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.131:32931 (local address 127.0.0.1:60784)
0809 09:33:30.302197 (+ 855us) negotiation.cc:107] Waiting for socket to connect
0809 09:33:30.302254 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0809 09:33:30.302382 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:33:30.303269 (+ 887us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:33:30.303272 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:33:30.303291 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 09:33:30.303396 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:33:30.303402 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:33:30.304897 (+ 1495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:33:30.304900 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:33:30.306304 (+ 1404us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:33:30.306316 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:33:30.307987 (+ 1671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:33:30.307990 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:33:30.308267 (+ 277us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:33:30.308291 (+ 24us) client_negotiation.cc:769] Sending connection context
0809 09:33:30.308369 (+ 78us) client_negotiation.cc:240] Negotiation successful
0809 09:33:30.308439 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":663,"mutex_wait_us":91,"thread_start_us":98,"threads_started":1}
I20260809 09:33:30.308915 4534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:33:30.301474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60784 (local address 127.20.125.131:32931)
0809 09:33:30.302303 (+ 829us) server_negotiation.cc:206] Beginning negotiation
0809 09:33:30.302315 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 09:33:30.302340 (+ 25us) server_negotiation.cc:407] Connection header received
0809 09:33:30.302685 (+ 345us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:33:30.302691 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:33:30.302909 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:33:30.303070 (+ 161us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 09:33:30.303538 (+ 468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:33:30.304740 (+ 1202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:33:30.306455 (+ 1715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:33:30.307690 (+ 1235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:33:30.307810 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:33:30.307923 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0809 09:33:30.308572 (+ 649us) server_negotiation.cc:299] Negotiation successful
0809 09:33:30.308768 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":725,"thread_start_us":182,"threads_started":1}
I20260809 09:33:30.310690 4218 tablet_service.cc:1467] Processing CreateTablet for tablet 62afd3a4929b4175b497f4a6db9a18c9 (DEFAULT_TABLE table=test-table [id=8f79ef49752e4f039cb48dd15ac0aa8b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 09:33:30.311723 4218 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 62afd3a4929b4175b497f4a6db9a18c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 09:33:30.311385 4040 tablet_service.cc:1467] Processing CreateTablet for tablet 62afd3a4929b4175b497f4a6db9a18c9 (DEFAULT_TABLE table=test-table [id=8f79ef49752e4f039cb48dd15ac0aa8b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 09:33:30.311570 4417 tablet_service.cc:1467] Processing CreateTablet for tablet 62afd3a4929b4175b497f4a6db9a18c9 (DEFAULT_TABLE table=test-table [id=8f79ef49752e4f039cb48dd15ac0aa8b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 09:33:30.312489 4040 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 62afd3a4929b4175b497f4a6db9a18c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 09:33:30.312493 4417 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 62afd3a4929b4175b497f4a6db9a18c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 09:33:30.318384 4540 tablet_bootstrap.cc:492] T 62afd3a4929b4175b497f4a6db9a18c9 P 3f80f6138ee847598719f77e05a60190: Bootstrap starting.
I20260809 09:33:30.319984 4544 tablet_bootstrap.cc:492] T 62afd3a4929b4175b497f4a6db9a18c9 P b7bd860b388e4818b53dae80c142be08: Bootstrap starting.
I20260809 09:33:30.320636 4545 tablet_bootstrap.cc:492] T 62afd3a4929b4175b497f4a6db9a18c9 P f822542f73ca42d6aad42efbc11efdc5: Bootstrap starting.
I20260809 09:33:30.321586 4540 tablet_bootstrap.cc:654] T 62afd3a4929b4175b497f4a6db9a18c9 P 3f80f6138ee847598719f77e05a60190: Neither blocks nor log segments found. Creating new log.
I20260809 09:33:30.322208 4544 tablet_bootstrap.cc:654] T 62afd3a4929b4175b497f4a6db9a18c9 P b7bd860b388e4818b53dae80c142be08: Neither blocks nor log segments found. Creating new log.
I20260809 09:33:30.322538 4540 log.cc:824] T 62afd3a4929b4175b497f4a6db9a18c9 P 3f80f6138ee847598719f77e05a60190: Log is configured to *not* fsync() on all Append() calls
I20260809 09:33:30.323143 4545 tablet_bootstrap.cc:654] T 62afd3a4929b4175b497f4a6db9a18c9 P f822542f73ca42d6aad42efbc11efdc5: Neither blocks nor log segments found. Creating new log.
I20260809 09:33:30.323123 4544 log.cc:824] T 62afd3a4929b4175b497f4a6db9a18c9 P b7bd860b388e4818b53dae80c142be08: Log is configured to *not* fsync() on all Append() calls
I20260809 09:33:30.323952 4545 log.cc:824] T 62afd3a4929b4175b497f4a6db9a18c9 P f822542f73ca42d6aad42efbc11efdc5: Log is configured to *not* fsync() on all Append() calls
I20260809 09:33:30.324695 4540 tablet_bootstrap.cc:492] T 62afd3a4929b4175b497f4a6db9a18c9 P 3f80f6138ee847598719f77e05a60190: No bootstrap required, opened a new log
I20260809 09:33:30.324924 4540 ts_tablet_manager.cc:1397] T 62afd3a4929b4175b497f4a6db9a18c9 P 3f80f6138ee847598719f77e05a60190: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.002s
I20260809 09:33:30.325002 4544 tablet_bootstrap.cc:492] T 62afd3a4929b4175b497f4a6db9a18c9 P b7bd860b388e4818b53dae80c142be08: No bootstrap required, opened a new log
I20260809 09:33:30.325168 4544 ts_tablet_manager.cc:1397] T 62afd3a4929b4175b497f4a6db9a18c9 P b7bd860b388e4818b53dae80c142be08: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.002s
I20260809 09:33:30.325870 4545 tablet_bootstrap.cc:492] T 62afd3a4929b4175b497f4a6db9a18c9 P f822542f73ca42d6aad42efbc11efdc5: No bootstrap required, opened a new log
I20260809 09:33:30.326043 4545 ts_tablet_manager.cc:1397] T 62afd3a4929b4175b497f4a6db9a18c9 P f822542f73ca42d6aad42efbc11efdc5: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260809 09:33:30.328251 4540 raft_consensus.cc:348] T 62afd3a4929b4175b497f4a6db9a18c9 P 3f80f6138ee847598719f77e05a60190 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f822542f73ca42d6aad42efbc11efdc5" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 32931 } } peers { permanent_uuid: "3f80f6138ee847598719f77e05a60190" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 33091 } } peers { permanent_uuid: "b7bd860b388e4818b53dae80c142be08" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 34895 } }
I20260809 09:33:30.328496 4540 raft_consensus.cc:374] T 62afd3a4929b4175b497f4a6db9a18c9 P 3f80f6138ee847598719f77e05a60190 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 09:33:30.328553 4540 raft_consensus.cc:729] T 62afd3a4929b4175b497f4a6db9a18c9 P 3f80f6138ee847598719f77e05a60190 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f80f6138ee847598719f77e05a60190, State: Initialized, Role: FOLLOWER
I20260809 09:33:30.329015 4540 consensus_queue.cc:260] T 62afd3a4929b4175b497f4a6db9a18c9 P 3f80f6138ee847598719f77e05a60190 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f822542f73ca42d6aad42efbc11efdc5" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 32931 } } peers { permanent_uuid: "3f80f6138ee847598719f77e05a60190" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 33091 } } peers { permanent_uuid: "b7bd860b388e4818b53dae80c142be08" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 34895 } }
I20260809 09:33:30.329542 4544 raft_consensus.cc:348] T 62afd3a4929b4175b497f4a6db9a18c9 P b7bd860b388e4818b53dae80c142be08 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f822542f73ca42d6aad42efbc11efdc5" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 32931 } } peers { permanent_uuid: "3f80f6138ee847598719f77e05a60190" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 33091 } } peers { permanent_uuid: "b7bd860b388e4818b53dae80c142be08" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 34895 } }
I20260809 09:33:30.329733 4316 heartbeater.cc:503] Master 127.20.125.190:33643 was elected leader, sending a full tablet report...
I20260809 09:33:30.329816 4544 raft_consensus.cc:374] T 62afd3a4929b4175b497f4a6db9a18c9 P b7bd860b388e4818b53dae80c142be08 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 09:33:30.329869 4544 raft_consensus.cc:729] T 62afd3a4929b4175b497f4a6db9a18c9 P b7bd860b388e4818b53dae80c142be08 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b7bd860b388e4818b53dae80c142be08, State: Initialized, Role: FOLLOWER
I20260809 09:33:30.330075 4540 ts_tablet_manager.cc:1428] T 62afd3a4929b4175b497f4a6db9a18c9 P 3f80f6138ee847598719f77e05a60190: Time spent starting tablet: real 0.005s user 0.005s sys 0.001s
I20260809 09:33:30.330263 4544 consensus_queue.cc:260] T 62afd3a4929b4175b497f4a6db9a18c9 P b7bd860b388e4818b53dae80c142be08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f822542f73ca42d6aad42efbc11efdc5" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 32931 } } peers { permanent_uuid: "3f80f6138ee847598719f77e05a60190" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 33091 } } peers { permanent_uuid: "b7bd860b388e4818b53dae80c142be08" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 34895 } }
I20260809 09:33:30.330510 4545 raft_consensus.cc:348] T 62afd3a4929b4175b497f4a6db9a18c9 P f822542f73ca42d6aad42efbc11efdc5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f822542f73ca42d6aad42efbc11efdc5" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 32931 } } peers { permanent_uuid: "3f80f6138ee847598719f77e05a60190" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 33091 } } peers { permanent_uuid: "b7bd860b388e4818b53dae80c142be08" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 34895 } }
I20260809 09:33:30.330763 4545 raft_consensus.cc:374] T 62afd3a4929b4175b497f4a6db9a18c9 P f822542f73ca42d6aad42efbc11efdc5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 09:33:30.330823 4545 raft_consensus.cc:729] T 62afd3a4929b4175b497f4a6db9a18c9 P f822542f73ca42d6aad42efbc11efdc5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f822542f73ca42d6aad42efbc11efdc5, State: Initialized, Role: FOLLOWER
I20260809 09:33:30.331063 4133 heartbeater.cc:503] Master 127.20.125.190:33643 was elected leader, sending a full tablet report...
I20260809 09:33:30.331240 4545 consensus_queue.cc:260] T 62afd3a4929b4175b497f4a6db9a18c9 P f822542f73ca42d6aad42efbc11efdc5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f822542f73ca42d6aad42efbc11efdc5" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 32931 } } peers { permanent_uuid: "3f80f6138ee847598719f77e05a60190" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 33091 } } peers { permanent_uuid: "b7bd860b388e4818b53dae80c142be08" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 34895 } }
I20260809 09:33:30.331643 4544 ts_tablet_manager.cc:1428] T 62afd3a4929b4175b497f4a6db9a18c9 P b7bd860b388e4818b53dae80c142be08: Time spent starting tablet: real 0.006s user 0.005s sys 0.002s
I20260809 09:33:30.332101 4545 ts_tablet_manager.cc:1428] T 62afd3a4929b4175b497f4a6db9a18c9 P f822542f73ca42d6aad42efbc11efdc5: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260809 09:33:30.332762 4518 heartbeater.cc:503] Master 127.20.125.190:33643 was elected leader, sending a full tablet report...
I20260809 09:33:30.361882 4554 raft_consensus.cc:482] T 62afd3a4929b4175b497f4a6db9a18c9 P b7bd860b388e4818b53dae80c142be08 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 09:33:30.362139 4554 raft_consensus.cc:504] T 62afd3a4929b4175b497f4a6db9a18c9 P b7bd860b388e4818b53dae80c142be08 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f822542f73ca42d6aad42efbc11efdc5" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 32931 } } peers { permanent_uuid: "3f80f6138ee847598719f77e05a60190" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 33091 } } peers { permanent_uuid: "b7bd860b388e4818b53dae80c142be08" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 34895 } }
I20260809 09:33:30.363409 4554 leader_election.cc:288] T 62afd3a4929b4175b497f4a6db9a18c9 P b7bd860b388e4818b53dae80c142be08 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f822542f73ca42d6aad42efbc11efdc5 (127.20.125.131:32931), 3f80f6138ee847598719f77e05a60190 (127.20.125.130:33091)
I20260809 09:33:30.367810 4534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:33:30.363749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.129:33983 (local address 127.20.125.131:32931)
0809 09:33:30.363934 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0809 09:33:30.363940 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 09:33:30.364249 (+ 309us) server_negotiation.cc:407] Connection header received
0809 09:33:30.364498 (+ 249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:33:30.364502 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:33:30.364555 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:33:30.364655 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 09:33:30.365136 (+ 481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:33:30.365635 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:33:30.366806 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:33:30.367211 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:33:30.367356 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:33:30.367411 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0809 09:33:30.367587 (+ 176us) server_negotiation.cc:299] Negotiation successful
0809 09:33:30.367675 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260809 09:33:30.367839 4561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:33:30.363620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.131:32931 (local address 127.20.125.129:33983)
0809 09:33:30.364150 (+ 530us) negotiation.cc:107] Waiting for socket to connect
0809 09:33:30.364166 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 09:33:30.364338 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:33:30.364802 (+ 464us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:33:30.364805 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:33:30.364846 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 09:33:30.365018 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:33:30.365025 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:33:30.365753 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:33:30.365758 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:33:30.366681 (+ 923us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:33:30.366692 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:33:30.367356 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:33:30.367360 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:33:30.367463 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:33:30.367483 (+ 20us) client_negotiation.cc:769] Sending connection context
0809 09:33:30.367572 (+ 89us) client_negotiation.cc:240] Negotiation successful
0809 09:33:30.367675 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":456,"mutex_wait_us":25,"spinlock_wait_cycles":768,"thread_start_us":141,"threads_started":1}
I20260809 09:33:30.367987 4536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:33:30.363845 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.129:44701 (local address 127.20.125.130:33091)
0809 09:33:30.363984 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0809 09:33:30.363989 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 09:33:30.364276 (+ 287us) server_negotiation.cc:407] Connection header received
0809 09:33:30.364485 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:33:30.364490 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:33:30.364549 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:33:30.364663 (+ 114us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 09:33:30.365119 (+ 456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:33:30.365701 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:33:30.366916 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:33:30.367374 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:33:30.367519 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:33:30.367576 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0809 09:33:30.367789 (+ 213us) server_negotiation.cc:299] Negotiation successful
0809 09:33:30.367868 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260809 09:33:30.368011 4556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:33:30.363741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.130:33091 (local address 127.20.125.129:44701)
0809 09:33:30.364195 (+ 454us) negotiation.cc:107] Waiting for socket to connect
0809 09:33:30.364209 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 09:33:30.364340 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:33:30.364727 (+ 387us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:33:30.364730 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:33:30.364757 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 09:33:30.364985 (+ 228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:33:30.364993 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:33:30.365834 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:33:30.365838 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:33:30.366781 (+ 943us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:33:30.366792 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:33:30.367508 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:33:30.367512 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:33:30.367652 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:33:30.367672 (+ 20us) client_negotiation.cc:769] Sending connection context
0809 09:33:30.367771 (+ 99us) client_negotiation.cc:240] Negotiation successful
0809 09:33:30.367843 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":385,"mutex_wait_us":56,"spinlock_wait_cycles":41216,"thread_start_us":117,"threads_started":1}
I20260809 09:33:30.368481 4449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62afd3a4929b4175b497f4a6db9a18c9" candidate_uuid: "b7bd860b388e4818b53dae80c142be08" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f822542f73ca42d6aad42efbc11efdc5" is_pre_election: true
I20260809 09:33:30.368539 4247 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62afd3a4929b4175b497f4a6db9a18c9" candidate_uuid: "b7bd860b388e4818b53dae80c142be08" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f80f6138ee847598719f77e05a60190" is_pre_election: true
I20260809 09:33:30.368829 4247 raft_consensus.cc:2454] T 62afd3a4929b4175b497f4a6db9a18c9 P 3f80f6138ee847598719f77e05a60190 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b7bd860b388e4818b53dae80c142be08 in term 0.
I20260809 09:33:30.368829 4449 raft_consensus.cc:2454] T 62afd3a4929b4175b497f4a6db9a18c9 P f822542f73ca42d6aad42efbc11efdc5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b7bd860b388e4818b53dae80c142be08 in term 0.
I20260809 09:33:30.369377 3975 leader_election.cc:302] T 62afd3a4929b4175b497f4a6db9a18c9 P b7bd860b388e4818b53dae80c142be08 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 3f80f6138ee847598719f77e05a60190, b7bd860b388e4818b53dae80c142be08, f822542f73ca42d6aad42efbc11efdc5; no voters:
I20260809 09:33:30.369753 4554 raft_consensus.cc:2781] T 62afd3a4929b4175b497f4a6db9a18c9 P b7bd860b388e4818b53dae80c142be08 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 09:33:30.369837 4554 raft_consensus.cc:482] T 62afd3a4929b4175b497f4a6db9a18c9 P b7bd860b388e4818b53dae80c142be08 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 09:33:30.369868 4554 raft_consensus.cc:3037] T 62afd3a4929b4175b497f4a6db9a18c9 P b7bd860b388e4818b53dae80c142be08 [term 0 FOLLOWER]: Advancing to term 1
I20260809 09:33:30.371002 4554 raft_consensus.cc:504] T 62afd3a4929b4175b497f4a6db9a18c9 P b7bd860b388e4818b53dae80c142be08 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f822542f73ca42d6aad42efbc11efdc5" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 32931 } } peers { permanent_uuid: "3f80f6138ee847598719f77e05a60190" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 33091 } } peers { permanent_uuid: "b7bd860b388e4818b53dae80c142be08" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 34895 } }
I20260809 09:33:30.371400 4554 leader_election.cc:288] T 62afd3a4929b4175b497f4a6db9a18c9 P b7bd860b388e4818b53dae80c142be08 [CANDIDATE]: Term 1 election: Requested vote from peers f822542f73ca42d6aad42efbc11efdc5 (127.20.125.131:32931), 3f80f6138ee847598719f77e05a60190 (127.20.125.130:33091)
I20260809 09:33:30.372133 4247 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62afd3a4929b4175b497f4a6db9a18c9" candidate_uuid: "b7bd860b388e4818b53dae80c142be08" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f80f6138ee847598719f77e05a60190"
I20260809 09:33:30.372133 4449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62afd3a4929b4175b497f4a6db9a18c9" candidate_uuid: "b7bd860b388e4818b53dae80c142be08" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f822542f73ca42d6aad42efbc11efdc5"
I20260809 09:33:30.372282 4449 raft_consensus.cc:3037] T 62afd3a4929b4175b497f4a6db9a18c9 P f822542f73ca42d6aad42efbc11efdc5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 09:33:30.372282 4247 raft_consensus.cc:3037] T 62afd3a4929b4175b497f4a6db9a18c9 P 3f80f6138ee847598719f77e05a60190 [term 0 FOLLOWER]: Advancing to term 1
I20260809 09:33:30.373795 4247 raft_consensus.cc:2454] T 62afd3a4929b4175b497f4a6db9a18c9 P 3f80f6138ee847598719f77e05a60190 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b7bd860b388e4818b53dae80c142be08 in term 1.
I20260809 09:33:30.373795 4449 raft_consensus.cc:2454] T 62afd3a4929b4175b497f4a6db9a18c9 P f822542f73ca42d6aad42efbc11efdc5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b7bd860b388e4818b53dae80c142be08 in term 1.
I20260809 09:33:30.374161 3975 leader_election.cc:302] T 62afd3a4929b4175b497f4a6db9a18c9 P b7bd860b388e4818b53dae80c142be08 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f80f6138ee847598719f77e05a60190, b7bd860b388e4818b53dae80c142be08; no voters:
I20260809 09:33:30.374614 4554 raft_consensus.cc:2781] T 62afd3a4929b4175b497f4a6db9a18c9 P b7bd860b388e4818b53dae80c142be08 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 09:33:30.374962 4554 raft_consensus.cc:686] T 62afd3a4929b4175b497f4a6db9a18c9 P b7bd860b388e4818b53dae80c142be08 [term 1 LEADER]: Becoming Leader. State: Replica: b7bd860b388e4818b53dae80c142be08, State: Running, Role: LEADER
I20260809 09:33:30.375459 4554 consensus_queue.cc:237] T 62afd3a4929b4175b497f4a6db9a18c9 P b7bd860b388e4818b53dae80c142be08 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f822542f73ca42d6aad42efbc11efdc5" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 32931 } } peers { permanent_uuid: "3f80f6138ee847598719f77e05a60190" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 33091 } } peers { permanent_uuid: "b7bd860b388e4818b53dae80c142be08" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 34895 } }
I20260809 09:33:30.379426 3862 catalog_manager.cc:5495] T 62afd3a4929b4175b497f4a6db9a18c9 P b7bd860b388e4818b53dae80c142be08 reported cstate change: term changed from 0 to 1, leader changed from <none> to b7bd860b388e4818b53dae80c142be08 (127.20.125.129). New cstate: current_term: 1 leader_uuid: "b7bd860b388e4818b53dae80c142be08" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f822542f73ca42d6aad42efbc11efdc5" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 32931 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3f80f6138ee847598719f77e05a60190" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 33091 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b7bd860b388e4818b53dae80c142be08" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 34895 } health_report { overall_health: HEALTHY } } }
I20260809 09:33:30.408903 4539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:33:30.405110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48462 (local address 127.20.125.129:34895)
0809 09:33:30.405261 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0809 09:33:30.405266 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 09:33:30.405281 (+ 15us) server_negotiation.cc:407] Connection header received
0809 09:33:30.405408 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:33:30.405411 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:33:30.405468 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:33:30.405587 (+ 119us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 09:33:30.405890 (+ 303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:33:30.406436 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:33:30.407379 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:33:30.407601 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:33:30.407720 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:33:30.408060 (+ 340us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 09:33:30.408487 (+ 427us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 09:33:30.408631 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0809 09:33:30.408694 (+ 63us) server_negotiation.cc:299] Negotiation successful
0809 09:33:30.408751 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260809 09:33:30.415223 4449 raft_consensus.cc:1261] T 62afd3a4929b4175b497f4a6db9a18c9 P f822542f73ca42d6aad42efbc11efdc5 [term 1 FOLLOWER]: Refusing update from remote peer b7bd860b388e4818b53dae80c142be08: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 09:33:30.415223 4247 raft_consensus.cc:1261] T 62afd3a4929b4175b497f4a6db9a18c9 P 3f80f6138ee847598719f77e05a60190 [term 1 FOLLOWER]: Refusing update from remote peer b7bd860b388e4818b53dae80c142be08: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 09:33:30.416115 4562 consensus_queue.cc:1035] T 62afd3a4929b4175b497f4a6db9a18c9 P b7bd860b388e4818b53dae80c142be08 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f80f6138ee847598719f77e05a60190" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 33091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 09:33:30.416326 4554 consensus_queue.cc:1035] T 62afd3a4929b4175b497f4a6db9a18c9 P b7bd860b388e4818b53dae80c142be08 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f822542f73ca42d6aad42efbc11efdc5" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 32931 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 09:33:30.427042 4567 mvcc.cc:205] Tried to move back new op lower bound from 7316553770651897856 to 7316553770501193728. Current Snapshot: MvccSnapshot[applied={T|T < 7316553770651897856}]
I20260809 09:33:30.433482 3864 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38300:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 09:33:30.433693 3864 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38300:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 09:33:30.436604 3864 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7531f1facac5426986b4c85c1903c3f9: Sending DeleteTablet for 3 replicas of tablet 62afd3a4929b4175b497f4a6db9a18c9
I20260809 09:33:30.437529 4417 tablet_service.cc:1514] Processing DeleteTablet for tablet 62afd3a4929b4175b497f4a6db9a18c9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 09:33:30 UTC) from {username='kudu', principal='kudu/127.20.125.190@KRBTEST.COM'} at 127.0.0.1:60784
I20260809 09:33:30.437685 4040 tablet_service.cc:1514] Processing DeleteTablet for tablet 62afd3a4929b4175b497f4a6db9a18c9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 09:33:30 UTC) from {username='kudu', principal='kudu/127.20.125.190@KRBTEST.COM'} at 127.0.0.1:48446
I20260809 09:33:30.437690 4218 tablet_service.cc:1514] Processing DeleteTablet for tablet 62afd3a4929b4175b497f4a6db9a18c9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 09:33:30 UTC) from {username='kudu', principal='kudu/127.20.125.190@KRBTEST.COM'} at 127.0.0.1:40256
I20260809 09:33:30.438336 4580 tablet_replica.cc:330] T 62afd3a4929b4175b497f4a6db9a18c9 P f822542f73ca42d6aad42efbc11efdc5: stopping tablet replica
I20260809 09:33:30.438678 4584 tablet_replica.cc:330] T 62afd3a4929b4175b497f4a6db9a18c9 P 3f80f6138ee847598719f77e05a60190: stopping tablet replica
I20260809 09:33:30.438745 4580 raft_consensus.cc:2229] T 62afd3a4929b4175b497f4a6db9a18c9 P f822542f73ca42d6aad42efbc11efdc5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 09:33:30.438992 20982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu with pid 3946
I20260809 09:33:30.439213 4584 raft_consensus.cc:2229] T 62afd3a4929b4175b497f4a6db9a18c9 P 3f80f6138ee847598719f77e05a60190 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 09:33:30.439666 4584 raft_consensus.cc:2258] T 62afd3a4929b4175b497f4a6db9a18c9 P 3f80f6138ee847598719f77e05a60190 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 09:33:30.455987 4580 raft_consensus.cc:2258] T 62afd3a4929b4175b497f4a6db9a18c9 P f822542f73ca42d6aad42efbc11efdc5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 09:33:30.458885 4580 ts_tablet_manager.cc:1905] T 62afd3a4929b4175b497f4a6db9a18c9 P f822542f73ca42d6aad42efbc11efdc5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 09:33:30.462479 4580 ts_tablet_manager.cc:1918] T 62afd3a4929b4175b497f4a6db9a18c9 P f822542f73ca42d6aad42efbc11efdc5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 09:33:30.462597 4580 log.cc:1196] T 62afd3a4929b4175b497f4a6db9a18c9 P f822542f73ca42d6aad42efbc11efdc5: Deleting WAL directory at /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786267941944493-20982-0/minicluster-data/ts-2/wal/wals/62afd3a4929b4175b497f4a6db9a18c9
I20260809 09:33:30.462941 4580 ts_tablet_manager.cc:1939] T 62afd3a4929b4175b497f4a6db9a18c9 P f822542f73ca42d6aad42efbc11efdc5: Deleting consensus metadata
I20260809 09:33:30.464396 3845 catalog_manager.cc:4841] TS f822542f73ca42d6aad42efbc11efdc5 (127.20.125.131:32931): tablet 62afd3a4929b4175b497f4a6db9a18c9 (table test-table [id=8f79ef49752e4f039cb48dd15ac0aa8b]) successfully deleted
I20260809 09:33:30.476269 4584 ts_tablet_manager.cc:1905] T 62afd3a4929b4175b497f4a6db9a18c9 P 3f80f6138ee847598719f77e05a60190: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 09:33:30.479096 4584 ts_tablet_manager.cc:1918] T 62afd3a4929b4175b497f4a6db9a18c9 P 3f80f6138ee847598719f77e05a60190: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 09:33:30.479215 4584 log.cc:1196] T 62afd3a4929b4175b497f4a6db9a18c9 P 3f80f6138ee847598719f77e05a60190: Deleting WAL directory at /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786267941944493-20982-0/minicluster-data/ts-1/wal/wals/62afd3a4929b4175b497f4a6db9a18c9
I20260809 09:33:30.479552 4584 ts_tablet_manager.cc:1939] T 62afd3a4929b4175b497f4a6db9a18c9 P 3f80f6138ee847598719f77e05a60190: Deleting consensus metadata
I20260809 09:33:30.480821 3844 catalog_manager.cc:4841] TS 3f80f6138ee847598719f77e05a60190 (127.20.125.130:33091): tablet 62afd3a4929b4175b497f4a6db9a18c9 (table test-table [id=8f79ef49752e4f039cb48dd15ac0aa8b]) successfully deleted
W20260809 09:33:30.483537 3844 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.20.125.129:34895 (error 108)
I20260809 09:33:30.483588 20982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu with pid 4141
I20260809 09:33:30.495750 4532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:33:30.495371 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.129:34895 (local address 127.0.0.1:48468)
0809 09:33:30.495559 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0809 09:33:30.495646 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.129:34895: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260809 09:33:30.496026 3844 catalog_manager.cc:4584] TS b7bd860b388e4818b53dae80c142be08 (127.20.125.129:34895): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 62afd3a4929b4175b497f4a6db9a18c9: Network error: Client connection negotiation failed: client connection to 127.20.125.129:34895: connect: Connection refused (error 111)
I20260809 09:33:30.501683 20982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu with pid 4323
I20260809 09:33:30.513842 20982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu with pid 3820
2026-08-09T09:33:30Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2725 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786267941944493-20982-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 09:33:30 dist-test-slave-7kzw krb5kdc[4588](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 09:33:30 dist-test-slave-7kzw krb5kdc[4588](info): set up 2 sockets
Aug 09 09:33:30 dist-test-slave-7kzw krb5kdc[4588](info): commencing operation
krb5kdc: starting...
W20260809 09:33:31.749325 20982 mini_kdc.cc:121] Time spent starting KDC: real 1.204s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786267941944493-20982-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786267941944493-20982-0/krb5kdc/test-user.keytab.
Aug 09 09:33:31 dist-test-slave-7kzw krb5kdc[4588](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268011, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T09:33:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T09:33:31Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.20.125.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.20.125.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.20.125.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786267941944493-20982-0/krb5kdc/rangeradmin_127.20.125.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.20.125.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786267941944493-20982-0/krb5kdc/rangeradmin_127.20.125.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.20.125.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.20.125.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.20.125.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786267941944493-20982-0/krb5kdc/rangerlookup_127.20.125.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.20.125.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786267941944493-20982-0/krb5kdc/rangerlookup_127.20.125.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.20.125.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.20.125.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.20.125.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786267941944493-20982-0/krb5kdc/HTTP_127.20.125.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.20.125.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786267941944493-20982-0/krb5kdc/HTTP_127.20.125.148@KRBTEST.COM.keytab.
I20260809 09:33:31.936934 20982 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786267941944493-20982-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskYHpNFG/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786267941944493-20982-0/minicluster-data/postgres -l logfile start
2026-08-09 09:33:34.565 UTC [4627] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-09 09:33:34.565 UTC [4627] LOG: listening on IPv4 address "127.20.125.148", port 40645
2026-08-09 09:33:34.569 UTC [4627] LOG: listening on Unix socket "/tmp/.s.PGSQL.40645"
2026-08-09 09:33:34.592 UTC [4630] LOG: database system was shut down at 2026-08-09 09:33:33 UTC
2026-08-09 09:33:34.596 UTC [4627] LOG: database system is ready to accept connections
I20260809 09:33:35.754944 20982 mini_postgres.cc:96] Postgres bound to 40645
2026-08-09 09:33:35.764 UTC [4642] FATAL: database "slave" does not exist
127.20.125.148:40645 - accepting connections
I20260809 09:33:35.765460 20982 mini_ranger.cc:161] Starting Ranger...
I20260809 09:33:35.784559 20982 mini_ranger.cc:84] Created miniranger Postgres user
I20260809 09:33:36.337661 20982 mini_ranger.cc:87] Created ranger Postgres database
I20260809 09:33:36.337767 20982 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786267941944493-20982-0/minicluster-data/ranger-admin
2026-08-09 09:33:36,663 [I] DB FLAVOR :POSTGRES
2026-08-09 09:33:36,663 [I] --------- Verifying Ranger DB connection ---------
2026-08-09 09:33:36,663 [I] Checking connection..
2026-08-09 09:33:36,663 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYHpNFG/build/debug/bin/postgresql.jar:/tmp/dist-test-taskYHpNFG/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.125.148:40645/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-09 09:33:36,938 [I] Checking connection passed.
2026-08-09 09:33:36,938 [I] --------- Verifying version history table ---------
2026-08-09 09:33:36,939 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYHpNFG/build/debug/bin/postgresql.jar:/tmp/dist-test-taskYHpNFG/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.125.148:40645/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 09:33:37,219 [I] Table x_db_version_h does not exist in database ranger
2026-08-09 09:33:37,220 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYHpNFG/build/debug/bin/postgresql.jar:/tmp/dist-test-taskYHpNFG/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.125.148:40645/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 09:33:37,468 [I] Table x_db_version_h does not exist in database ranger
2026-08-09 09:33:37,468 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-09 09:33:37,468 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYHpNFG/build/debug/bin/postgresql.jar:/tmp/dist-test-taskYHpNFG/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.125.148:40645/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskYHpNFG/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-09 09:33:37.720 UTC [4737] WARNING: there is no transaction in progress
2026-08-09 09:33:37,729 [I] create_dbversion_catalog.sql file imported successfully
2026-08-09 09:33:37,730 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYHpNFG/build/debug/bin/postgresql.jar:/tmp/dist-test-taskYHpNFG/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.125.148:40645/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 09:33:38,001 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-09 09:33:38,002 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-09 09:33:38,002 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYHpNFG/build/debug/bin/postgresql.jar:/tmp/dist-test-taskYHpNFG/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.125.148:40645/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-09 09:33:38,258 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYHpNFG/build/debug/bin/postgresql.jar:/tmp/dist-test-taskYHpNFG/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.125.148:40645/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-09 09:33:38,531 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYHpNFG/build/debug/bin/postgresql.jar:/tmp/dist-test-taskYHpNFG/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.125.148:40645/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-7kzw.c.gcp-upstream.internal','N') ;"
2026-08-09 09:33:38,790 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYHpNFG/build/debug/bin/postgresql.jar:/tmp/dist-test-taskYHpNFG/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.125.148:40645/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-09 09:33:39,062 [I] Table x_portal_user does not exist in database ranger
2026-08-09 09:33:39,062 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYHpNFG/build/debug/bin/postgresql.jar:/tmp/dist-test-taskYHpNFG/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.125.148:40645/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-09 09:33:39,304 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-09 09:33:39,305 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYHpNFG/build/debug/bin/postgresql.jar:/tmp/dist-test-taskYHpNFG/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.125.148:40645/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-09 09:33:39,585 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYHpNFG/build/debug/bin/postgresql.jar:/tmp/dist-test-taskYHpNFG/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.125.148:40645/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-09 09:33:39,830 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-09 09:33:39,830 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYHpNFG/build/debug/bin/postgresql.jar:/tmp/dist-test-taskYHpNFG/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.125.148:40645/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskYHpNFG/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-09 09:33:40.865 UTC [4894] WARNING: there is no transaction in progress
2026-08-09 09:33:40.882 UTC [4894] WARNING: there is no transaction in progress
2026-08-09 09:33:40.900 UTC [4894] WARNING: there is no transaction in progress
2026-08-09 09:33:40.916 UTC [4894] WARNING: there is no transaction in progress
2026-08-09 09:33:40.933 UTC [4894] WARNING: there is no transaction in progress
2026-08-09 09:33:40.950 UTC [4894] WARNING: there is no transaction in progress
2026-08-09 09:33:41.012 UTC [4894] WARNING: there is no transaction in progress
2026-08-09 09:33:41.020 UTC [4894] WARNING: there is no transaction in progress
2026-08-09 09:33:41.035 UTC [4894] WARNING: there is no transaction in progress
2026-08-09 09:33:41.046 UTC [4894] WARNING: there is no transaction in progress
2026-08-09 09:33:41.057 UTC [4894] WARNING: there is no transaction in progress
2026-08-09 09:33:41.073 UTC [4894] WARNING: there is no transaction in progress
2026-08-09 09:33:41.083 UTC [4894] WARNING: there is no transaction in progress
2026-08-09 09:33:41.090 UTC [4894] WARNING: there is no transaction in progress
2026-08-09 09:33:41.098 UTC [4894] WARNING: there is no transaction in progress
2026-08-09 09:33:41.105 UTC [4894] WARNING: there is no transaction in progress
2026-08-09 09:33:41,782 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-09 09:33:41,783 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYHpNFG/build/debug/bin/postgresql.jar:/tmp/dist-test-taskYHpNFG/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.125.148:40645/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-09 09:33:42,027 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-09 09:33:42,027 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYHpNFG/build/debug/bin/postgresql.jar:/tmp/dist-test-taskYHpNFG/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.125.148:40645/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-09 09:33:42,300 [I] Table x_portal_user already exists in database 'ranger'
2026-08-09 09:33:42,300 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYHpNFG/build/debug/bin/postgresql.jar:/tmp/dist-test-taskYHpNFG/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.125.148:40645/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-09 09:33:42,558 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-09 09:33:42,558 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYHpNFG/build/debug/bin/postgresql.jar:/tmp/dist-test-taskYHpNFG/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.125.148:40645/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-09 09:33:42,795 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYHpNFG/build/debug/bin/postgresql.jar:/tmp/dist-test-taskYHpNFG/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.125.148:40645/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-09 09:33:43,076 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYHpNFG/build/debug/bin/postgresql.jar:/tmp/dist-test-taskYHpNFG/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.125.148:40645/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-7kzw.c.gcp-upstream.internal';"
2026-08-09 09:33:43,345 [I] CORE_DB_SCHEMA import status has been updated
2026-08-09 09:33:43,346 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYHpNFG/build/debug/bin/postgresql.jar:/tmp/dist-test-taskYHpNFG/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.125.148:40645/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-09 09:33:43,601 [I] DB_PATCHES have already been applied
I20260809 09:33:43.607743 20982 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786267941944493-20982-0/minicluster-data/ranger-admin:/tmp/dist-test-taskYHpNFG/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskYHpNFG/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskYHpNFG/build/debug/bin/postgresql.jar:/tmp/dist-test-taskYHpNFG/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260809 09:33:43.607833 20982 mini_ranger.cc:193] Using host: 127.20.125.148
I20260809 09:33:43.612618 20982 mini_ranger.cc:227] Ranger admin URL: http://127.20.125.148:57079
Aug 09, 2026 9:33:43 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 09, 2026 9:33:43 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 09, 2026 9:33:44 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786267941944493-20982-0/minicluster-data/ranger-admin/ews/webapp
Aug 09, 2026 9:33:44 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786267941944493-20982-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 09, 2026 9:33:44 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786267941944493-20982-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 09, 2026 9:33:44 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786267941944493-20982-0/minicluster-data/ranger-admin/ews/webapp].
Aug 09, 2026 9:33:44 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260809 09:33:45.458983 20982 mini_ranger.cc:160] Time spent starting Ranger: real 9.694s user 0.000s sys 0.013s
Aug 09, 2026 9:34:05 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 09, 2026 9:34:06 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Aug 09, 2026 9:34:06 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 09, 2026 9:34:06 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 09, 2026 9:34:06 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 09, 2026 9:34:06 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 09, 2026 9:34:06 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 09, 2026 9:34:06 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 09, 2026 9:34:06 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 09, 2026 9:34:06 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 09, 2026 9:34:06 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 09, 2026 9:34:06 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 09, 2026 9:34:06 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 09, 2026 9:34:06 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 09, 2026 9:34:06 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 09, 2026 9:34:06 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 09, 2026 9:34:06 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 09, 2026 9:34:07 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260809 09:34:07.685352 20982 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.20.125.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.20.125.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.20.125.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786267941944493-20982-0/krb5kdc/rangerkms_127.20.125.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.20.125.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786267941944493-20982-0/krb5kdc/rangerkms_127.20.125.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.20.125.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.20.125.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.20.125.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786267941944493-20982-0/krb5kdc/HTTP_127.20.125.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.20.125.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786267941944493-20982-0/krb5kdc/HTTP_127.20.125.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 09 09:34:07 dist-test-slave-7kzw krb5kdc[4588](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268047, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260809 09:34:07.789270 20982 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260809 09:34:07.801949 20982 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260809 09:34:08.712730 20982 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260809 09:34:08.712839 20982 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786267941944493-20982-0/minicluster-data/ranger-kms
I20260809 09:34:08.712953 20982 mini_ranger_kms.cc:227] Using postgres at 127.20.125.148:40645
2026-08-09 09:34:09,166 [I] DB FLAVOR :POSTGRES
2026-08-09 09:34:09,167 [I] --------- Verifying Ranger DB connection ---------
2026-08-09 09:34:09,167 [I] Checking connection
2026-08-09 09:34:09,167 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYHpNFG/build/debug/bin/postgresql.jar:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786267941944493-20982-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.125.148:40645/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-09 09:34:09,457 [I] connection success
2026-08-09 09:34:09,457 [I] --------- Verifying Ranger DB tables ---------
2026-08-09 09:34:09,457 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-09 09:34:09,457 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYHpNFG/build/debug/bin/postgresql.jar:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786267941944493-20982-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.125.148:40645/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-09 09:34:09,738 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-09 09:34:09,738 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-09 09:34:09,739 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-09 09:34:09,739 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYHpNFG/build/debug/bin/postgresql.jar:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786267941944493-20982-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.125.148:40645/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786267941944493-20982-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-09 09:34:10,036 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260809 09:34:10.590828 20982 mini_ranger_kms.cc:310] Created kms service
I20260809 09:34:10.796343 20982 mini_ranger_kms.cc:326] Created kudu user
I20260809 09:34:10.843902 20982 mini_ranger_kms.cc:343] Created rangerkms user
Aug 09 09:34:10 dist-test-slave-7kzw krb5kdc[4588](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268047, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.20.125.148@KRBTEST.COM
I20260809 09:34:11.271077 20982 mini_ranger_kms.cc:384] Added ranger policy
I20260809 09:34:11.271427 20982 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786267941944493-20982-0/minicluster-data/ranger-kms:/tmp/dist-test-taskYHpNFG/build/debug/bin/postgresql.jar:/tmp/dist-test-taskYHpNFG/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskYHpNFG/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260809 09:34:11.271463 20982 mini_ranger_kms.cc:240] Using host: 127.20.125.148
I20260809 09:34:11.276557 20982 mini_ranger_kms.cc:276] Ranger KMS PID: 5174
I20260809 09:34:11.276697 20982 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.20.125.148:36841
Aug 09, 2026 9:34:12 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 09, 2026 9:34:12 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 09, 2026 9:34:12 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786267941944493-20982-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 09, 2026 9:34:12 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786267941944493-20982-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 09, 2026 9:34:13 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786267941944493-20982-0/minicluster-data/ranger-kms/ews/webapp].
Aug 09, 2026 9:34:13 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 09 09:34:13 dist-test-slave-7kzw krb5kdc[4588](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786268053, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.20.125.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 09:34:13.955463 20982 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.166s user 0.002s sys 0.428s
I20260809 09:34:13.955623 20982 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260809 09:34:13.955739 20982 mini_ranger_kms.cc:401] 127.20.125.148:36841/kms/v1/keys
Aug 09 09:34:14 dist-test-slave-7kzw krb5kdc[4588](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268053, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.20.125.148@KRBTEST.COM for HTTP/127.20.125.148@KRBTEST.COM
Aug 09 09:34:14 dist-test-slave-7kzw krb5kdc[4588](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268053, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.20.125.148@KRBTEST.COM for HTTP/127.20.125.148@KRBTEST.COM
Aug 09 09:34:16 dist-test-slave-7kzw krb5kdc[4588](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268056, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.20.125.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.125.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.125.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.190.keytab.
Entry for principal kudu/127.20.125.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.190.keytab.
I20260809 09:34:17.041360 20982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
/tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786267941944493-20982-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786267941944493-20982-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786267941944493-20982-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786267941944493-20982-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.125.190:41921
--webserver_interface=127.20.125.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.190.keytab
--principal=kudu/127.20.125.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.125.148:36773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.125.190:41921
--ranger_config_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786267941944493-20982-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.20.125.148:36841/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786267941944493-20982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786267941944493-20982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786267941944493-20982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 09:34:17.180971 5246 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 09:34:17.181236 5246 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 09:34:17.181281 5246 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 09:34:17.185199 5246 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 09:34:17.185250 5246 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 09:34:17.185269 5246 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 09:34:17.185285 5246 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 09:34:17.185300 5246 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 09:34:17.190737 5246 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.125.148:36773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786267941944493-20982-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786267941944493-20982-0/minicluster-data/master-0/wal
--ranger_kms_url=127.20.125.148:36841/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.20.125.190:41921
--ranger_config_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786267941944493-20982-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.190.keytab
--principal=kudu/127.20.125.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.125.190:41921
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786267941944493-20982-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.125.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5246
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786267941944493-20982-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 08:38:55 UTC on ccycloud.quasar-njjleu.root.comops.site
build id 4575935
I20260809 09:34:17.191671 5246 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 09:34:17.192615 5246 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 09:34:17.201777 5252 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:34:17.201908 5255 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:34:17.201969 5246 server_base.cc:1034] running on GCE node
W20260809 09:34:17.201839 5253 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:34:17.202741 5246 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 09:34:17.203871 5246 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 09:34:17.205037 5246 hybrid_clock.cc:648] HybridClock initialized: now 1786268057205023 us; error 27 us; skew 500 ppm
Aug 09 09:34:17 dist-test-slave-7kzw krb5kdc[4588](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268057, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 09:34:17.211490 5246 init.cc:377] Logged in from keytab as kudu/127.20.125.190@KRBTEST.COM (short username kudu)
I20260809 09:34:17.212435 5246 webserver.cc:460] Webserver started at http://127.20.125.190:33451/ using document root <none> and password file <none>
I20260809 09:34:17.213043 5246 fs_manager.cc:359] Metadata directory not provided
I20260809 09:34:17.213107 5246 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 09:34:17.213356 5246 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 09:34:17 dist-test-slave-7kzw krb5kdc[4588](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268057, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.190@KRBTEST.COM for HTTP/127.20.125.148@KRBTEST.COM
I20260809 09:34:17.276782 5246 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786267941944493-20982-0/minicluster-data/master-0/data/instance:
uuid: "04886f1a37334005842e45ca4685463b"
format_stamp: "Formatted at 2026-08-09 09:34:17 on dist-test-slave-7kzw"
server_key: "2b654c1dbecd50582231c9d5ee33b261"
server_key_iv: "da1a94cd25d641e7642489d839eefdc0"
server_key_version: "kuduclusterkey@0"
I20260809 09:34:17.277565 5246 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786267941944493-20982-0/minicluster-data/master-0/wal/instance:
uuid: "04886f1a37334005842e45ca4685463b"
format_stamp: "Formatted at 2026-08-09 09:34:17 on dist-test-slave-7kzw"
server_key: "2b654c1dbecd50582231c9d5ee33b261"
server_key_iv: "da1a94cd25d641e7642489d839eefdc0"
server_key_version: "kuduclusterkey@0"
I20260809 09:34:17.282181 5246 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260809 09:34:17.301964 5265 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 09:34:17.303529 5246 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 09:34:17.303674 5246 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786267941944493-20982-0/minicluster-data/master-0/data,/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786267941944493-20982-0/minicluster-data/master-0/wal
uuid: "04886f1a37334005842e45ca4685463b"
format_stamp: "Formatted at 2026-08-09 09:34:17 on dist-test-slave-7kzw"
server_key: "2b654c1dbecd50582231c9d5ee33b261"
server_key_iv: "da1a94cd25d641e7642489d839eefdc0"
server_key_version: "kuduclusterkey@0"
I20260809 09:34:17.303815 5246 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786267941944493-20982-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786267941944493-20982-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786267941944493-20982-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 09:34:17.319110 5246 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 09:34:17.322378 5246 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 09:34:17.322600 5246 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 09:34:17.336887 5246 rpc_server.cc:307] RPC server started. Bound to: 127.20.125.190:41921
I20260809 09:34:17.337042 5351 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.125.190:41921 every 8 connection(s)
I20260809 09:34:17.338125 5246 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786267941944493-20982-0/minicluster-data/master-0/data/info.pb
I20260809 09:34:17.342178 20982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu as pid 5246
I20260809 09:34:17.342206 5352 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 09:34:17.342337 20982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786267941944493-20982-0/minicluster-data/master-0/wal/instance
I20260809 09:34:17.342730 20982 external_mini_cluster.cc:1428] Setting key 014f663794e77a72081be3ffc419984b
I20260809 09:34:17.351183 5352 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 04886f1a37334005842e45ca4685463b: Bootstrap starting.
Aug 09 09:34:17 dist-test-slave-7kzw krb5kdc[4588](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268056, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.125.190@KRBTEST.COM
I20260809 09:34:17.354163 5352 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 04886f1a37334005842e45ca4685463b: Neither blocks nor log segments found. Creating new log.
I20260809 09:34:17.355345 5352 log.cc:824] T 00000000000000000000000000000000 P 04886f1a37334005842e45ca4685463b: Log is configured to *not* fsync() on all Append() calls
I20260809 09:34:17.357781 5352 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 04886f1a37334005842e45ca4685463b: No bootstrap required, opened a new log
I20260809 09:34:17.359575 5355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:34:17.344399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60302 (local address 127.20.125.190:41921)
0809 09:34:17.345102 (+ 703us) server_negotiation.cc:206] Beginning negotiation
0809 09:34:17.345147 (+ 45us) server_negotiation.cc:399] Waiting for connection header
0809 09:34:17.345188 (+ 41us) server_negotiation.cc:407] Connection header received
0809 09:34:17.346302 (+ 1114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:34:17.346324 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:34:17.346744 (+ 420us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:34:17.347097 (+ 353us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:34:17.348029 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:34:17.349108 (+ 1079us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:34:17.350019 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:34:17.350572 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:34:17.350751 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:34:17.353582 (+ 2831us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:34:17.353588 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:34:17.353602 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:34:17.353631 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:34:17.356391 (+ 2760us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:34:17.356890 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:34:17.356897 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:34:17.356909 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:34:17.357039 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:34:17.357368 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:34:17.357372 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:34:17.357373 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:34:17.357783 (+ 410us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:34:17.358005 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0809 09:34:17.358427 (+ 422us) server_negotiation.cc:299] Negotiation successful
0809 09:34:17.358796 (+ 369us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":489,"thread_start_us":236,"threads_started":1}
I20260809 09:34:17.362112 5352 raft_consensus.cc:348] T 00000000000000000000000000000000 P 04886f1a37334005842e45ca4685463b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04886f1a37334005842e45ca4685463b" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 41921 } }
I20260809 09:34:17.362404 5352 raft_consensus.cc:374] T 00000000000000000000000000000000 P 04886f1a37334005842e45ca4685463b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 09:34:17.362455 5352 raft_consensus.cc:729] T 00000000000000000000000000000000 P 04886f1a37334005842e45ca4685463b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 04886f1a37334005842e45ca4685463b, State: Initialized, Role: FOLLOWER
I20260809 09:34:17.362949 5352 consensus_queue.cc:260] T 00000000000000000000000000000000 P 04886f1a37334005842e45ca4685463b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04886f1a37334005842e45ca4685463b" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 41921 } }
I20260809 09:34:17.363118 5352 raft_consensus.cc:388] T 00000000000000000000000000000000 P 04886f1a37334005842e45ca4685463b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 09:34:17.363183 5352 raft_consensus.cc:482] T 00000000000000000000000000000000 P 04886f1a37334005842e45ca4685463b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 09:34:17.363278 5352 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 04886f1a37334005842e45ca4685463b [term 0 FOLLOWER]: Advancing to term 1
I20260809 09:34:17.364514 5352 raft_consensus.cc:504] T 00000000000000000000000000000000 P 04886f1a37334005842e45ca4685463b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04886f1a37334005842e45ca4685463b" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 41921 } }
I20260809 09:34:17.364893 5352 leader_election.cc:302] T 00000000000000000000000000000000 P 04886f1a37334005842e45ca4685463b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 04886f1a37334005842e45ca4685463b; no voters:
I20260809 09:34:17.365270 5352 leader_election.cc:288] T 00000000000000000000000000000000 P 04886f1a37334005842e45ca4685463b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 09:34:17.365461 5357 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 04886f1a37334005842e45ca4685463b [term 1 FOLLOWER]: Leader election won for term 1
I20260809 09:34:17.365898 5357 raft_consensus.cc:686] T 00000000000000000000000000000000 P 04886f1a37334005842e45ca4685463b [term 1 LEADER]: Becoming Leader. State: Replica: 04886f1a37334005842e45ca4685463b, State: Running, Role: LEADER
I20260809 09:34:17.366304 5357 consensus_queue.cc:237] T 00000000000000000000000000000000 P 04886f1a37334005842e45ca4685463b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04886f1a37334005842e45ca4685463b" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 41921 } }
I20260809 09:34:17.366780 5352 sys_catalog.cc:564] T 00000000000000000000000000000000 P 04886f1a37334005842e45ca4685463b [sys.catalog]: configured and running, proceeding with master startup.
I20260809 09:34:17.367538 5357 sys_catalog.cc:455] T 00000000000000000000000000000000 P 04886f1a37334005842e45ca4685463b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "04886f1a37334005842e45ca4685463b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04886f1a37334005842e45ca4685463b" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 41921 } } }
I20260809 09:34:17.367744 5357 sys_catalog.cc:458] T 00000000000000000000000000000000 P 04886f1a37334005842e45ca4685463b [sys.catalog]: This master's current role is: LEADER
I20260809 09:34:17.368131 5357 sys_catalog.cc:455] T 00000000000000000000000000000000 P 04886f1a37334005842e45ca4685463b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 04886f1a37334005842e45ca4685463b. Latest consensus state: current_term: 1 leader_uuid: "04886f1a37334005842e45ca4685463b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04886f1a37334005842e45ca4685463b" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 41921 } } }
I20260809 09:34:17.368237 5357 sys_catalog.cc:458] T 00000000000000000000000000000000 P 04886f1a37334005842e45ca4685463b [sys.catalog]: This master's current role is: LEADER
I20260809 09:34:17.370669 5352 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786267941944493-20982-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260809 09:34:17.371160 5370 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 09:34:17.378966 5370 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 09:34:17.389042 5370 catalog_manager.cc:1347] Generated new cluster ID: 4df6d694c9114138aa1064a3b6382178
I20260809 09:34:17.389137 5370 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 09:34:17.422714 5370 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 09:34:17.424312 5370 catalog_manager.cc:1504] Loading token signing keys...
I20260809 09:34:17.435139 5370 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 04886f1a37334005842e45ca4685463b: Generated new TSK 0
I20260809 09:34:17.436232 5370 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.20.125.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.125.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.125.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.129.keytab.
Entry for principal kudu/127.20.125.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.129.keytab.
I20260809 09:34:18.225301 20982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
/tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786267941944493-20982-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786267941944493-20982-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786267941944493-20982-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786267941944493-20982-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.20.125.148:36841/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.125.129:0
--local_ip_for_outbound_sockets=127.20.125.129
--webserver_interface=127.20.125.129
--webserver_port=0
--tserver_master_addrs=127.20.125.190:41921
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.129.keytab
--principal=kudu/127.20.125.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.125.148:36773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786267941944493-20982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786267941944493-20982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786267941944493-20982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 09:34:18.369019 5419 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 09:34:18.369300 5419 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 09:34:18.369414 5419 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 09:34:18.374279 5419 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 09:34:18.374447 5419 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 09:34:18.374621 5419 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.125.129
I20260809 09:34:18.381233 5419 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.125.148:36773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786267941944493-20982-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786267941944493-20982-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.20.125.148:36841/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.129.keytab
--principal=kudu/127.20.125.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.125.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786267941944493-20982-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.125.129
--webserver_port=0
--tserver_master_addrs=127.20.125.190:41921
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5419
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.125.129
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786267941944493-20982-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 08:38:55 UTC on ccycloud.quasar-njjleu.root.comops.site
build id 4575935
I20260809 09:34:18.382685 5419 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 09:34:18.384243 5419 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 09:34:18.403107 5430 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:34:18.404568 5429 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:34:18.404852 5432 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:34:18.406638 5419 server_base.cc:1034] running on GCE node
I20260809 09:34:18.407229 5419 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 09:34:18.408165 5419 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 09:34:18.409452 5419 hybrid_clock.cc:648] HybridClock initialized: now 1786268058409427 us; error 34 us; skew 500 ppm
Aug 09 09:34:18 dist-test-slave-7kzw krb5kdc[4588](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268058, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 09:34:18.413230 5419 init.cc:377] Logged in from keytab as kudu/127.20.125.129@KRBTEST.COM (short username kudu)
I20260809 09:34:18.414208 5419 webserver.cc:460] Webserver started at http://127.20.125.129:36785/ using document root <none> and password file <none>
I20260809 09:34:18.414919 5419 fs_manager.cc:359] Metadata directory not provided
I20260809 09:34:18.414987 5419 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 09:34:18.415218 5419 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 09:34:18 dist-test-slave-7kzw krb5kdc[4588](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268058, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.129@KRBTEST.COM for HTTP/127.20.125.148@KRBTEST.COM
I20260809 09:34:18.437134 5419 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786267941944493-20982-0/minicluster-data/ts-0/data/instance:
uuid: "fdf79434349b4f6aa8e0cfc7174d0421"
format_stamp: "Formatted at 2026-08-09 09:34:18 on dist-test-slave-7kzw"
server_key: "35f73b3012eda82574c370ce7ff0bf46"
server_key_iv: "bf7ef55362e9d27320cbba3b415523c0"
server_key_version: "kuduclusterkey@0"
I20260809 09:34:18.438084 5419 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786267941944493-20982-0/minicluster-data/ts-0/wal/instance:
uuid: "fdf79434349b4f6aa8e0cfc7174d0421"
format_stamp: "Formatted at 2026-08-09 09:34:18 on dist-test-slave-7kzw"
server_key: "35f73b3012eda82574c370ce7ff0bf46"
server_key_iv: "bf7ef55362e9d27320cbba3b415523c0"
server_key_version: "kuduclusterkey@0"
I20260809 09:34:18.443641 5419 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 09:34:18.475579 5442 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 09:34:18.477509 5419 fs_manager.cc:725] Time spent opening block manager: real 0.017s user 0.003s sys 0.000s
I20260809 09:34:18.477748 5419 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786267941944493-20982-0/minicluster-data/ts-0/data,/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786267941944493-20982-0/minicluster-data/ts-0/wal
uuid: "fdf79434349b4f6aa8e0cfc7174d0421"
format_stamp: "Formatted at 2026-08-09 09:34:18 on dist-test-slave-7kzw"
server_key: "35f73b3012eda82574c370ce7ff0bf46"
server_key_iv: "bf7ef55362e9d27320cbba3b415523c0"
server_key_version: "kuduclusterkey@0"
I20260809 09:34:18.477994 5419 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786267941944493-20982-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786267941944493-20982-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786267941944493-20982-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 09:34:18.498785 5419 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 09:34:18.502565 5419 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 09:34:18.502730 5419 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 09:34:18.503296 5419 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 09:34:18.504688 5419 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 09:34:18.505043 5419 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260809 09:34:18.505183 5419 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 09:34:18.505250 5419 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 09:34:18.531876 5419 rpc_server.cc:307] RPC server started. Bound to: 127.20.125.129:46169
I20260809 09:34:18.533422 5419 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786267941944493-20982-0/minicluster-data/ts-0/data/info.pb
I20260809 09:34:18.534261 5616 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.125.129:46169 every 8 connection(s)
I20260809 09:34:18.537318 20982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu as pid 5419
I20260809 09:34:18.537451 20982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786267941944493-20982-0/minicluster-data/ts-0/wal/instance
I20260809 09:34:18.537774 20982 external_mini_cluster.cc:1428] Setting key 1fdd111a38c7820f5ee95ae455da956c
Aug 09 09:34:18 dist-test-slave-7kzw krb5kdc[4588](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268058, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.129@KRBTEST.COM for kudu/127.20.125.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.20.125.130@KRBTEST.COM; defaulting to no policy
I20260809 09:34:18.559142 5621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:34:18.537511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.129:54115 (local address 127.20.125.190:41921)
0809 09:34:18.537857 (+ 346us) server_negotiation.cc:206] Beginning negotiation
0809 09:34:18.537862 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 09:34:18.542959 (+ 5097us) server_negotiation.cc:407] Connection header received
0809 09:34:18.544604 (+ 1645us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:34:18.544608 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:34:18.544669 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:34:18.544756 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:34:18.546992 (+ 2236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:34:18.547873 (+ 881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:34:18.549756 (+ 1883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:34:18.550109 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:34:18.550192 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:34:18.553721 (+ 3529us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:34:18.553725 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:34:18.553727 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:34:18.553758 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:34:18.556010 (+ 2252us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:34:18.556783 (+ 773us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:34:18.556788 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:34:18.556790 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:34:18.556904 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:34:18.557403 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:34:18.557409 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:34:18.557414 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:34:18.557673 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:34:18.557785 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0809 09:34:18.558792 (+ 1007us) server_negotiation.cc:299] Negotiation successful
0809 09:34:18.558941 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":240,"thread_start_us":99,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.125.130@KRBTEST.COM" created.
I20260809 09:34:18.561518 5623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:34:18.537988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:41921 (local address 127.20.125.129:54115)
0809 09:34:18.542794 (+ 4806us) negotiation.cc:107] Waiting for socket to connect
0809 09:34:18.542859 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0809 09:34:18.544179 (+ 1320us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:34:18.545225 (+ 1046us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:34:18.545236 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:34:18.545895 (+ 659us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:34:18.546797 (+ 902us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:34:18.546820 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:34:18.548045 (+ 1225us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:34:18.548048 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:34:18.549599 (+ 1551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:34:18.549611 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:34:18.550433 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:34:18.550437 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:34:18.550684 (+ 247us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:34:18.551432 (+ 748us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 09:34:18.551455 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:34:18.553415 (+ 1960us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 09:34:18.556162 (+ 2747us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:34:18.556170 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:34:18.556188 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:34:18.556631 (+ 443us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:34:18.557040 (+ 409us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:34:18.557043 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:34:18.557045 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:34:18.557257 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:34:18.557847 (+ 590us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 09:34:18.557890 (+ 43us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 09:34:18.558275 (+ 385us) client_negotiation.cc:769] Sending connection context
0809 09:34:18.559891 (+ 1616us) client_negotiation.cc:240] Negotiation successful
0809 09:34:18.560195 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4552,"thread_start_us":160,"threads_started":1}
I20260809 09:34:18.563185 5619 heartbeater.cc:347] Connected to a master server at 127.20.125.190:41921
I20260809 09:34:18.563506 5619 heartbeater.cc:464] Registering TS with master...
I20260809 09:34:18.564121 5619 heartbeater.cc:511] Master 127.20.125.190:41921 requested a full tablet report, sending...
I20260809 09:34:18.566021 5292 ts_manager.cc:194] Registered new tserver with Master: fdf79434349b4f6aa8e0cfc7174d0421 (127.20.125.129:46169)
I20260809 09:34:18.567912 5292 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.125.129@KRBTEST.COM'} at 127.20.125.129:54115
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.125.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.130.keytab.
Entry for principal kudu/127.20.125.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.130.keytab.
I20260809 09:34:18.581810 20982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
/tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786267941944493-20982-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786267941944493-20982-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786267941944493-20982-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786267941944493-20982-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.20.125.148:36841/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.125.130:0
--local_ip_for_outbound_sockets=127.20.125.130
--webserver_interface=127.20.125.130
--webserver_port=0
--tserver_master_addrs=127.20.125.190:41921
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.130.keytab
--principal=kudu/127.20.125.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.125.148:36773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786267941944493-20982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786267941944493-20982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786267941944493-20982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 09:34:18 dist-test-slave-7kzw krb5kdc[4588](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786268058, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260809 09:34:18.750356 5627 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 09:34:18.750903 5627 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 09:34:18.751000 5627 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 09:34:18.757154 5627 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 09:34:18.757302 5627 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 09:34:18.757440 5627 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.125.130
I20260809 09:34:18.765734 5627 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.125.148:36773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786267941944493-20982-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786267941944493-20982-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.20.125.148:36841/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.130.keytab
--principal=kudu/127.20.125.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.125.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786267941944493-20982-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.125.130
--webserver_port=0
--tserver_master_addrs=127.20.125.190:41921
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5627
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.125.130
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786267941944493-20982-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 08:38:55 UTC on ccycloud.quasar-njjleu.root.comops.site
build id 4575935
I20260809 09:34:18.767264 5627 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 09:34:18.768776 5627 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 09:34:18.783309 5638 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:34:18.783593 5636 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:34:18.783344 5635 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:34:18.785171 5627 server_base.cc:1034] running on GCE node
I20260809 09:34:18.785737 5627 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 09:34:18.786636 5627 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 09:34:18.787840 5627 hybrid_clock.cc:648] HybridClock initialized: now 1786268058787810 us; error 45 us; skew 500 ppm
Aug 09 09:34:18 dist-test-slave-7kzw krb5kdc[4588](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268058, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 09:34:18.792230 5627 init.cc:377] Logged in from keytab as kudu/127.20.125.130@KRBTEST.COM (short username kudu)
I20260809 09:34:18.793567 5627 webserver.cc:460] Webserver started at http://127.20.125.130:44237/ using document root <none> and password file <none>
I20260809 09:34:18.794492 5627 fs_manager.cc:359] Metadata directory not provided
I20260809 09:34:18.794590 5627 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 09:34:18.794874 5627 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 09:34:18 dist-test-slave-7kzw krb5kdc[4588](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268058, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.130@KRBTEST.COM for HTTP/127.20.125.148@KRBTEST.COM
I20260809 09:34:18.814109 5627 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786267941944493-20982-0/minicluster-data/ts-1/data/instance:
uuid: "e61f59be12f343cb8a03d3433b9816d6"
format_stamp: "Formatted at 2026-08-09 09:34:18 on dist-test-slave-7kzw"
server_key: "9c85534ac8625c2f73dc3e7f4aa6e318"
server_key_iv: "f8edeff7c9e7f7e88befdeffd96e7d36"
server_key_version: "kuduclusterkey@0"
I20260809 09:34:18.815184 5627 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786267941944493-20982-0/minicluster-data/ts-1/wal/instance:
uuid: "e61f59be12f343cb8a03d3433b9816d6"
format_stamp: "Formatted at 2026-08-09 09:34:18 on dist-test-slave-7kzw"
server_key: "9c85534ac8625c2f73dc3e7f4aa6e318"
server_key_iv: "f8edeff7c9e7f7e88befdeffd96e7d36"
server_key_version: "kuduclusterkey@0"
I20260809 09:34:18.821823 5627 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260809 09:34:18.843842 5649 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 09:34:18.845568 5627 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260809 09:34:18.845737 5627 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786267941944493-20982-0/minicluster-data/ts-1/data,/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786267941944493-20982-0/minicluster-data/ts-1/wal
uuid: "e61f59be12f343cb8a03d3433b9816d6"
format_stamp: "Formatted at 2026-08-09 09:34:18 on dist-test-slave-7kzw"
server_key: "9c85534ac8625c2f73dc3e7f4aa6e318"
server_key_iv: "f8edeff7c9e7f7e88befdeffd96e7d36"
server_key_version: "kuduclusterkey@0"
I20260809 09:34:18.845897 5627 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786267941944493-20982-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786267941944493-20982-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786267941944493-20982-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 09:34:18.881170 5627 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 09:34:18.888429 5627 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 09:34:18.888677 5627 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 09:34:18.889511 5627 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 09:34:18.890801 5627 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 09:34:18.890880 5627 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 09:34:18.890944 5627 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 09:34:18.890983 5627 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 09:34:18.907058 5627 rpc_server.cc:307] RPC server started. Bound to: 127.20.125.130:45849
I20260809 09:34:18.908370 5627 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786267941944493-20982-0/minicluster-data/ts-1/data/info.pb
I20260809 09:34:18.912460 5812 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.125.130:45849 every 8 connection(s)
I20260809 09:34:18.916558 20982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu as pid 5627
I20260809 09:34:18.916695 20982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786267941944493-20982-0/minicluster-data/ts-1/wal/instance
I20260809 09:34:18.917026 20982 external_mini_cluster.cc:1428] Setting key b6af7960e248760559f61455608cc932
Aug 09 09:34:18 dist-test-slave-7kzw krb5kdc[4588](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268058, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.130@KRBTEST.COM for kudu/127.20.125.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.20.125.131@KRBTEST.COM; defaulting to no policy
I20260809 09:34:18.934805 5621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:34:18.914614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.130:48553 (local address 127.20.125.190:41921)
0809 09:34:18.914818 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0809 09:34:18.914824 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 09:34:18.917641 (+ 2817us) server_negotiation.cc:407] Connection header received
0809 09:34:18.917714 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:34:18.917717 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:34:18.917785 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:34:18.917892 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:34:18.920138 (+ 2246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:34:18.921180 (+ 1042us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:34:18.923571 (+ 2391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:34:18.924007 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:34:18.924122 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:34:18.928663 (+ 4541us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:34:18.928667 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:34:18.928671 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:34:18.928777 (+ 106us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:34:18.931319 (+ 2542us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:34:18.932112 (+ 793us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:34:18.932128 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:34:18.932130 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:34:18.932244 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:34:18.932722 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:34:18.932725 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:34:18.932727 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:34:18.933043 (+ 316us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:34:18.933175 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0809 09:34:18.934205 (+ 1030us) server_negotiation.cc:299] Negotiation successful
0809 09:34:18.934433 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.125.131@KRBTEST.COM" created.
I20260809 09:34:18.937045 5815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:34:18.915033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:41921 (local address 127.20.125.130:48553)
0809 09:34:18.916079 (+ 1046us) negotiation.cc:107] Waiting for socket to connect
0809 09:34:18.916125 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 09:34:18.917418 (+ 1293us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:34:18.918102 (+ 684us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:34:18.918113 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:34:18.918876 (+ 763us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:34:18.919903 (+ 1027us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:34:18.919930 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:34:18.921481 (+ 1551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:34:18.921486 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:34:18.923354 (+ 1868us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:34:18.923377 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:34:18.924414 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:34:18.924418 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:34:18.924662 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:34:18.925544 (+ 882us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 09:34:18.925569 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:34:18.928353 (+ 2784us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 09:34:18.931493 (+ 3140us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:34:18.931500 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:34:18.931519 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:34:18.931961 (+ 442us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:34:18.932374 (+ 413us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:34:18.932379 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:34:18.932381 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:34:18.932585 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:34:18.933284 (+ 699us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 09:34:18.933294 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 09:34:18.933746 (+ 452us) client_negotiation.cc:769] Sending connection context
0809 09:34:18.935204 (+ 1458us) client_negotiation.cc:240] Negotiation successful
0809 09:34:18.935614 (+ 410us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":861,"thread_start_us":144,"threads_started":1}
I20260809 09:34:18.941099 5813 heartbeater.cc:347] Connected to a master server at 127.20.125.190:41921
I20260809 09:34:18.941406 5813 heartbeater.cc:464] Registering TS with master...
I20260809 09:34:18.942242 5813 heartbeater.cc:511] Master 127.20.125.190:41921 requested a full tablet report, sending...
I20260809 09:34:18.944324 5292 ts_manager.cc:194] Registered new tserver with Master: e61f59be12f343cb8a03d3433b9816d6 (127.20.125.130:45849)
I20260809 09:34:18.945613 5292 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.125.130@KRBTEST.COM'} at 127.20.125.130:48553
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.125.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.131.keytab.
Entry for principal kudu/127.20.125.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.131.keytab.
I20260809 09:34:18.962781 20982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
/tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786267941944493-20982-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786267941944493-20982-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786267941944493-20982-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786267941944493-20982-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.20.125.148:36841/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.125.131:0
--local_ip_for_outbound_sockets=127.20.125.131
--webserver_interface=127.20.125.131
--webserver_port=0
--tserver_master_addrs=127.20.125.190:41921
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.131.keytab
--principal=kudu/127.20.125.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.125.148:36773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786267941944493-20982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786267941944493-20982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786267941944493-20982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 09:34:19.182413 5820 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 09:34:19.183046 5820 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 09:34:19.183209 5820 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 09:34:19.189847 5820 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 09:34:19.190019 5820 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 09:34:19.190189 5820 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.125.131
I20260809 09:34:19.199071 5820 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.125.148:36773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786267941944493-20982-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786267941944493-20982-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.20.125.148:36841/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.131.keytab
--principal=kudu/127.20.125.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.125.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786267941944493-20982-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.125.131
--webserver_port=0
--tserver_master_addrs=127.20.125.190:41921
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5820
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.125.131
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786267941944493-20982-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 08:38:55 UTC on ccycloud.quasar-njjleu.root.comops.site
build id 4575935
I20260809 09:34:19.201246 5820 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 09:34:19.203058 5820 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 09:34:19.220752 5829 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:34:19.221524 5832 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:34:19.221676 5827 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:34:19.221978 5820 server_base.cc:1034] running on GCE node
I20260809 09:34:19.222525 5820 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 09:34:19.223331 5820 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 09:34:19.230675 5820 hybrid_clock.cc:648] HybridClock initialized: now 1786268059230610 us; error 70 us; skew 500 ppm
Aug 09 09:34:19 dist-test-slave-7kzw krb5kdc[4588](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268059, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 09:34:19.234673 5820 init.cc:377] Logged in from keytab as kudu/127.20.125.131@KRBTEST.COM (short username kudu)
I20260809 09:34:19.235917 5820 webserver.cc:460] Webserver started at http://127.20.125.131:45019/ using document root <none> and password file <none>
I20260809 09:34:19.236763 5820 fs_manager.cc:359] Metadata directory not provided
I20260809 09:34:19.236918 5820 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 09:34:19.237222 5820 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 09:34:19 dist-test-slave-7kzw krb5kdc[4588](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268059, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.131@KRBTEST.COM for HTTP/127.20.125.148@KRBTEST.COM
Aug 09 09:34:19 dist-test-slave-7kzw krb5kdc[4588](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268058, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.190@KRBTEST.COM for HTTP/127.20.125.148@KRBTEST.COM
I20260809 09:34:19.260025 5820 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786267941944493-20982-0/minicluster-data/ts-2/data/instance:
uuid: "a42f7eccc4104d82ac636aea66ddea99"
format_stamp: "Formatted at 2026-08-09 09:34:19 on dist-test-slave-7kzw"
server_key: "27a022eebe83c465181850412a8db39a"
server_key_iv: "0c7dc6b00499fbbbc22cffe7516400f9"
server_key_version: "kuduclusterkey@0"
I20260809 09:34:19.261950 5820 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786267941944493-20982-0/minicluster-data/ts-2/wal/instance:
uuid: "a42f7eccc4104d82ac636aea66ddea99"
format_stamp: "Formatted at 2026-08-09 09:34:19 on dist-test-slave-7kzw"
server_key: "27a022eebe83c465181850412a8db39a"
server_key_iv: "0c7dc6b00499fbbbc22cffe7516400f9"
server_key_version: "kuduclusterkey@0"
I20260809 09:34:19.272722 5820 fs_manager.cc:691] Time spent creating directory manager: real 0.010s user 0.004s sys 0.000s
I20260809 09:34:19.311304 5841 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 09:34:19.313246 5820 fs_manager.cc:725] Time spent opening block manager: real 0.019s user 0.001s sys 0.000s
I20260809 09:34:19.313411 5820 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786267941944493-20982-0/minicluster-data/ts-2/data,/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786267941944493-20982-0/minicluster-data/ts-2/wal
uuid: "a42f7eccc4104d82ac636aea66ddea99"
format_stamp: "Formatted at 2026-08-09 09:34:19 on dist-test-slave-7kzw"
server_key: "27a022eebe83c465181850412a8db39a"
server_key_iv: "0c7dc6b00499fbbbc22cffe7516400f9"
server_key_version: "kuduclusterkey@0"
I20260809 09:34:19.313580 5820 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786267941944493-20982-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786267941944493-20982-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786267941944493-20982-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 09:34:19.345115 5820 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 09:34:19.349406 5820 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 09:34:19.349694 5820 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 09:34:19.350562 5820 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 09:34:19.351837 5820 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 09:34:19.352005 5820 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 09:34:19.352106 5820 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 09:34:19.352167 5820 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 09:34:19.373061 5820 rpc_server.cc:307] RPC server started. Bound to: 127.20.125.131:45957
I20260809 09:34:19.374310 5820 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786267941944493-20982-0/minicluster-data/ts-2/data/info.pb
I20260809 09:34:19.376845 20982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu as pid 5820
I20260809 09:34:19.376998 20982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786267941944493-20982-0/minicluster-data/ts-2/wal/instance
I20260809 09:34:19.377353 20982 external_mini_cluster.cc:1428] Setting key 0d8a08c494a9ee4f32327a6b00a799b0
I20260809 09:34:19.376669 6002 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.125.131:45957 every 8 connection(s)
Aug 09 09:34:19 dist-test-slave-7kzw krb5kdc[4588](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268059, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.131@KRBTEST.COM for kudu/127.20.125.190@KRBTEST.COM
I20260809 09:34:19.403258 5621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:34:19.382627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.131:57055 (local address 127.20.125.190:41921)
0809 09:34:19.382761 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0809 09:34:19.382766 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 09:34:19.383959 (+ 1193us) server_negotiation.cc:407] Connection header received
0809 09:34:19.385738 (+ 1779us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:34:19.385742 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:34:19.385801 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:34:19.385857 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:34:19.388388 (+ 2531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:34:19.389452 (+ 1064us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:34:19.392367 (+ 2915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:34:19.392930 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:34:19.393041 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:34:19.397065 (+ 4024us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:34:19.397071 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:34:19.397074 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:34:19.397106 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:34:19.399549 (+ 2443us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:34:19.400373 (+ 824us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:34:19.400378 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:34:19.400382 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:34:19.400484 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:34:19.401129 (+ 645us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:34:19.401134 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:34:19.401136 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:34:19.401436 (+ 300us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:34:19.401534 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0809 09:34:19.402917 (+ 1383us) server_negotiation.cc:299] Negotiation successful
0809 09:34:19.403067 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260809 09:34:19.405602 6005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:34:19.381397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:41921 (local address 127.20.125.131:57055)
0809 09:34:19.383749 (+ 2352us) negotiation.cc:107] Waiting for socket to connect
0809 09:34:19.383837 (+ 88us) client_negotiation.cc:174] Beginning negotiation
0809 09:34:19.385391 (+ 1554us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:34:19.386360 (+ 969us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:34:19.386371 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:34:19.387086 (+ 715us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:34:19.388194 (+ 1108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:34:19.388218 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:34:19.389660 (+ 1442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:34:19.389664 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:34:19.392124 (+ 2460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:34:19.392150 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:34:19.393296 (+ 1146us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:34:19.393302 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:34:19.393563 (+ 261us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:34:19.394457 (+ 894us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 09:34:19.394529 (+ 72us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:34:19.396847 (+ 2318us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 09:34:19.399765 (+ 2918us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:34:19.399783 (+ 18us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:34:19.399802 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:34:19.400225 (+ 423us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:34:19.400661 (+ 436us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:34:19.400675 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:34:19.400678 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:34:19.400948 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:34:19.401611 (+ 663us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 09:34:19.401620 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 09:34:19.402322 (+ 702us) client_negotiation.cc:769] Sending connection context
0809 09:34:19.403854 (+ 1532us) client_negotiation.cc:240] Negotiation successful
0809 09:34:19.404261 (+ 407us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2028,"thread_start_us":253,"threads_started":1}
I20260809 09:34:19.407511 6003 heartbeater.cc:347] Connected to a master server at 127.20.125.190:41921
I20260809 09:34:19.407830 6003 heartbeater.cc:464] Registering TS with master...
I20260809 09:34:19.408552 6003 heartbeater.cc:511] Master 127.20.125.190:41921 requested a full tablet report, sending...
I20260809 09:34:19.410079 5292 ts_manager.cc:194] Registered new tserver with Master: a42f7eccc4104d82ac636aea66ddea99 (127.20.125.131:45957)
I20260809 09:34:19.411010 5292 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.125.131@KRBTEST.COM'} at 127.20.125.131:57055
I20260809 09:34:19.415817 20982 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 09:34:19.437784 5621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:34:19.418982 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60318 (local address 127.20.125.190:41921)
0809 09:34:19.419154 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0809 09:34:19.419162 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 09:34:19.426910 (+ 7748us) server_negotiation.cc:407] Connection header received
0809 09:34:19.427144 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:34:19.427148 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:34:19.427206 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:34:19.427267 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:34:19.428633 (+ 1366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:34:19.429495 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:34:19.430927 (+ 1432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:34:19.431307 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:34:19.431394 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:34:19.433539 (+ 2145us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:34:19.433543 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:34:19.433546 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:34:19.433576 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:34:19.435696 (+ 2120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:34:19.436257 (+ 561us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:34:19.436261 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:34:19.436264 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:34:19.436362 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:34:19.436728 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:34:19.436732 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:34:19.436734 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:34:19.436958 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:34:19.437075 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0809 09:34:19.437459 (+ 384us) server_negotiation.cc:299] Negotiation successful
0809 09:34:19.437586 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 09:34:19.441995 5292 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60318:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 09:34:19.448164 5292 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 09:34:19.473410 6017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:34:19.463432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.129:46169 (local address 127.0.0.1:49912)
0809 09:34:19.465105 (+ 1673us) negotiation.cc:107] Waiting for socket to connect
0809 09:34:19.465162 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0809 09:34:19.465314 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:34:19.466537 (+ 1223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:34:19.466543 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:34:19.466576 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 09:34:19.466715 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:34:19.466723 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:34:19.468207 (+ 1484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:34:19.468211 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:34:19.470596 (+ 2385us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:34:19.470608 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:34:19.472055 (+ 1447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:34:19.472059 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:34:19.472942 (+ 883us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:34:19.472988 (+ 46us) client_negotiation.cc:769] Sending connection context
0809 09:34:19.473109 (+ 121us) client_negotiation.cc:240] Negotiation successful
0809 09:34:19.473247 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1488,"thread_start_us":122,"threads_started":1}
I20260809 09:34:19.474114 6019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:34:19.464306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49912 (local address 127.20.125.129:46169)
0809 09:34:19.464830 (+ 524us) server_negotiation.cc:206] Beginning negotiation
0809 09:34:19.464838 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 09:34:19.465469 (+ 631us) server_negotiation.cc:407] Connection header received
0809 09:34:19.465594 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:34:19.465600 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:34:19.465825 (+ 225us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:34:19.465986 (+ 161us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 09:34:19.466863 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:34:19.468088 (+ 1225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:34:19.470738 (+ 2650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:34:19.471638 (+ 900us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:34:19.471808 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:34:19.471950 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0809 09:34:19.473808 (+ 1858us) server_negotiation.cc:299] Negotiation successful
0809 09:34:19.473922 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":401,"thread_start_us":116,"threads_started":1}
I20260809 09:34:19.478096 5516 tablet_service.cc:1467] Processing CreateTablet for tablet e4583229efdb4354b416add4b5272822 (DEFAULT_TABLE table=test-table [id=555201544d5e426cadcac6ae958ad460]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 09:34:19.480626 5516 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e4583229efdb4354b416add4b5272822. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 09:34:19.490013 6022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:34:19.465210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.131:45957 (local address 127.0.0.1:49990)
0809 09:34:19.468331 (+ 3121us) negotiation.cc:107] Waiting for socket to connect
0809 09:34:19.468345 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 09:34:19.468432 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:34:19.484921 (+ 16489us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:34:19.484924 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:34:19.484944 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 09:34:19.485023 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:34:19.485028 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:34:19.486705 (+ 1677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:34:19.486708 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:34:19.487930 (+ 1222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:34:19.487941 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:34:19.489510 (+ 1569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:34:19.489513 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:34:19.489620 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:34:19.489649 (+ 29us) client_negotiation.cc:769] Sending connection context
0809 09:34:19.489729 (+ 80us) client_negotiation.cc:240] Negotiation successful
0809 09:34:19.489807 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3048,"thread_start_us":179,"threads_started":1}
I20260809 09:34:19.491954 6023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:34:19.465964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49990 (local address 127.20.125.131:45957)
0809 09:34:19.482801 (+ 16837us) server_negotiation.cc:206] Beginning negotiation
0809 09:34:19.482842 (+ 41us) server_negotiation.cc:399] Waiting for connection header
0809 09:34:19.482870 (+ 28us) server_negotiation.cc:407] Connection header received
0809 09:34:19.482951 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:34:19.482956 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:34:19.483153 (+ 197us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:34:19.483420 (+ 267us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 09:34:19.485161 (+ 1741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:34:19.486541 (+ 1380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:34:19.488172 (+ 1631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:34:19.489179 (+ 1007us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:34:19.489304 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:34:19.489439 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0809 09:34:19.491659 (+ 2220us) server_negotiation.cc:299] Negotiation successful
0809 09:34:19.491769 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16695,"thread_start_us":314,"threads_started":1}
I20260809 09:34:19.495335 6024 tablet_bootstrap.cc:492] T e4583229efdb4354b416add4b5272822 P fdf79434349b4f6aa8e0cfc7174d0421: Bootstrap starting.
I20260809 09:34:19.495433 5908 tablet_service.cc:1467] Processing CreateTablet for tablet e4583229efdb4354b416add4b5272822 (DEFAULT_TABLE table=test-table [id=555201544d5e426cadcac6ae958ad460]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 09:34:19.496627 5908 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e4583229efdb4354b416add4b5272822. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 09:34:19.499349 6024 tablet_bootstrap.cc:654] T e4583229efdb4354b416add4b5272822 P fdf79434349b4f6aa8e0cfc7174d0421: Neither blocks nor log segments found. Creating new log.
I20260809 09:34:19.500543 6024 log.cc:824] T e4583229efdb4354b416add4b5272822 P fdf79434349b4f6aa8e0cfc7174d0421: Log is configured to *not* fsync() on all Append() calls
I20260809 09:34:19.503160 6024 tablet_bootstrap.cc:492] T e4583229efdb4354b416add4b5272822 P fdf79434349b4f6aa8e0cfc7174d0421: No bootstrap required, opened a new log
I20260809 09:34:19.503387 6024 ts_tablet_manager.cc:1397] T e4583229efdb4354b416add4b5272822 P fdf79434349b4f6aa8e0cfc7174d0421: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260809 09:34:19.508285 6018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:34:19.463474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.130:45849 (local address 127.0.0.1:41386)
0809 09:34:19.466252 (+ 2778us) negotiation.cc:107] Waiting for socket to connect
0809 09:34:19.466267 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 09:34:19.466395 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:34:19.479183 (+ 12788us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:34:19.479186 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:34:19.479209 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 09:34:19.479320 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:34:19.479326 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:34:19.483426 (+ 4100us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:34:19.483431 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:34:19.484807 (+ 1376us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:34:19.484820 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:34:19.507792 (+ 22972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:34:19.507797 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:34:19.507930 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:34:19.507963 (+ 33us) client_negotiation.cc:769] Sending connection context
0809 09:34:19.508027 (+ 64us) client_negotiation.cc:240] Negotiation successful
0809 09:34:19.508122 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2710,"thread_start_us":124,"threads_started":1}
I20260809 09:34:19.509054 6020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:34:19.464449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41386 (local address 127.20.125.130:45849)
0809 09:34:19.478639 (+ 14190us) server_negotiation.cc:206] Beginning negotiation
0809 09:34:19.478647 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 09:34:19.478679 (+ 32us) server_negotiation.cc:407] Connection header received
0809 09:34:19.478767 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:34:19.478773 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:34:19.478997 (+ 224us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:34:19.479500 (+ 503us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 09:34:19.479660 (+ 160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:34:19.480871 (+ 1211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:34:19.490596 (+ 9725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:34:19.491500 (+ 904us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:34:19.508525 (+ 17025us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:34:19.508724 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0809 09:34:19.508790 (+ 66us) server_negotiation.cc:299] Negotiation successful
0809 09:34:19.508899 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4097,"thread_start_us":156,"threads_started":1}
I20260809 09:34:19.510183 6024 raft_consensus.cc:348] T e4583229efdb4354b416add4b5272822 P fdf79434349b4f6aa8e0cfc7174d0421 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e61f59be12f343cb8a03d3433b9816d6" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 45849 } } peers { permanent_uuid: "a42f7eccc4104d82ac636aea66ddea99" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 45957 } } peers { permanent_uuid: "fdf79434349b4f6aa8e0cfc7174d0421" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 46169 } }
I20260809 09:34:19.510491 6024 raft_consensus.cc:374] T e4583229efdb4354b416add4b5272822 P fdf79434349b4f6aa8e0cfc7174d0421 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 09:34:19.510547 6024 raft_consensus.cc:729] T e4583229efdb4354b416add4b5272822 P fdf79434349b4f6aa8e0cfc7174d0421 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fdf79434349b4f6aa8e0cfc7174d0421, State: Initialized, Role: FOLLOWER
I20260809 09:34:19.511011 6024 consensus_queue.cc:260] T e4583229efdb4354b416add4b5272822 P fdf79434349b4f6aa8e0cfc7174d0421 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e61f59be12f343cb8a03d3433b9816d6" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 45849 } } peers { permanent_uuid: "a42f7eccc4104d82ac636aea66ddea99" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 45957 } } peers { permanent_uuid: "fdf79434349b4f6aa8e0cfc7174d0421" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 46169 } }
I20260809 09:34:19.512235 6024 ts_tablet_manager.cc:1428] T e4583229efdb4354b416add4b5272822 P fdf79434349b4f6aa8e0cfc7174d0421: Time spent starting tablet: real 0.009s user 0.007s sys 0.000s
I20260809 09:34:19.512620 5619 heartbeater.cc:503] Master 127.20.125.190:41921 was elected leader, sending a full tablet report...
I20260809 09:34:19.512523 5712 tablet_service.cc:1467] Processing CreateTablet for tablet e4583229efdb4354b416add4b5272822 (DEFAULT_TABLE table=test-table [id=555201544d5e426cadcac6ae958ad460]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 09:34:19.513163 6026 tablet_bootstrap.cc:492] T e4583229efdb4354b416add4b5272822 P a42f7eccc4104d82ac636aea66ddea99: Bootstrap starting.
I20260809 09:34:19.515604 5712 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e4583229efdb4354b416add4b5272822. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 09:34:19.516077 6026 tablet_bootstrap.cc:654] T e4583229efdb4354b416add4b5272822 P a42f7eccc4104d82ac636aea66ddea99: Neither blocks nor log segments found. Creating new log.
I20260809 09:34:19.516999 6026 log.cc:824] T e4583229efdb4354b416add4b5272822 P a42f7eccc4104d82ac636aea66ddea99: Log is configured to *not* fsync() on all Append() calls
I20260809 09:34:19.520395 6026 tablet_bootstrap.cc:492] T e4583229efdb4354b416add4b5272822 P a42f7eccc4104d82ac636aea66ddea99: No bootstrap required, opened a new log
I20260809 09:34:19.520725 6026 ts_tablet_manager.cc:1397] T e4583229efdb4354b416add4b5272822 P a42f7eccc4104d82ac636aea66ddea99: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260809 09:34:19.527288 6029 tablet_bootstrap.cc:492] T e4583229efdb4354b416add4b5272822 P e61f59be12f343cb8a03d3433b9816d6: Bootstrap starting.
I20260809 09:34:19.527987 6026 raft_consensus.cc:348] T e4583229efdb4354b416add4b5272822 P a42f7eccc4104d82ac636aea66ddea99 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e61f59be12f343cb8a03d3433b9816d6" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 45849 } } peers { permanent_uuid: "a42f7eccc4104d82ac636aea66ddea99" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 45957 } } peers { permanent_uuid: "fdf79434349b4f6aa8e0cfc7174d0421" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 46169 } }
I20260809 09:34:19.528437 6026 raft_consensus.cc:374] T e4583229efdb4354b416add4b5272822 P a42f7eccc4104d82ac636aea66ddea99 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 09:34:19.528636 6026 raft_consensus.cc:729] T e4583229efdb4354b416add4b5272822 P a42f7eccc4104d82ac636aea66ddea99 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a42f7eccc4104d82ac636aea66ddea99, State: Initialized, Role: FOLLOWER
I20260809 09:34:19.529203 6026 consensus_queue.cc:260] T e4583229efdb4354b416add4b5272822 P a42f7eccc4104d82ac636aea66ddea99 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e61f59be12f343cb8a03d3433b9816d6" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 45849 } } peers { permanent_uuid: "a42f7eccc4104d82ac636aea66ddea99" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 45957 } } peers { permanent_uuid: "fdf79434349b4f6aa8e0cfc7174d0421" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 46169 } }
I20260809 09:34:19.530085 6029 tablet_bootstrap.cc:654] T e4583229efdb4354b416add4b5272822 P e61f59be12f343cb8a03d3433b9816d6: Neither blocks nor log segments found. Creating new log.
I20260809 09:34:19.530418 6026 ts_tablet_manager.cc:1428] T e4583229efdb4354b416add4b5272822 P a42f7eccc4104d82ac636aea66ddea99: Time spent starting tablet: real 0.010s user 0.000s sys 0.007s
I20260809 09:34:19.530849 6003 heartbeater.cc:503] Master 127.20.125.190:41921 was elected leader, sending a full tablet report...
I20260809 09:34:19.532358 6029 log.cc:824] T e4583229efdb4354b416add4b5272822 P e61f59be12f343cb8a03d3433b9816d6: Log is configured to *not* fsync() on all Append() calls
I20260809 09:34:19.534577 6029 tablet_bootstrap.cc:492] T e4583229efdb4354b416add4b5272822 P e61f59be12f343cb8a03d3433b9816d6: No bootstrap required, opened a new log
I20260809 09:34:19.534775 6029 ts_tablet_manager.cc:1397] T e4583229efdb4354b416add4b5272822 P e61f59be12f343cb8a03d3433b9816d6: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.001s
W20260809 09:34:19.536152 5620 tablet.cc:2369] T e4583229efdb4354b416add4b5272822 P fdf79434349b4f6aa8e0cfc7174d0421: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 09:34:19.541087 6029 raft_consensus.cc:348] T e4583229efdb4354b416add4b5272822 P e61f59be12f343cb8a03d3433b9816d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e61f59be12f343cb8a03d3433b9816d6" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 45849 } } peers { permanent_uuid: "a42f7eccc4104d82ac636aea66ddea99" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 45957 } } peers { permanent_uuid: "fdf79434349b4f6aa8e0cfc7174d0421" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 46169 } }
I20260809 09:34:19.541373 6029 raft_consensus.cc:374] T e4583229efdb4354b416add4b5272822 P e61f59be12f343cb8a03d3433b9816d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 09:34:19.541432 6029 raft_consensus.cc:729] T e4583229efdb4354b416add4b5272822 P e61f59be12f343cb8a03d3433b9816d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e61f59be12f343cb8a03d3433b9816d6, State: Initialized, Role: FOLLOWER
I20260809 09:34:19.543761 6029 consensus_queue.cc:260] T e4583229efdb4354b416add4b5272822 P e61f59be12f343cb8a03d3433b9816d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e61f59be12f343cb8a03d3433b9816d6" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 45849 } } peers { permanent_uuid: "a42f7eccc4104d82ac636aea66ddea99" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 45957 } } peers { permanent_uuid: "fdf79434349b4f6aa8e0cfc7174d0421" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 46169 } }
I20260809 09:34:19.544585 5813 heartbeater.cc:503] Master 127.20.125.190:41921 was elected leader, sending a full tablet report...
I20260809 09:34:19.545011 6029 ts_tablet_manager.cc:1428] T e4583229efdb4354b416add4b5272822 P e61f59be12f343cb8a03d3433b9816d6: Time spent starting tablet: real 0.010s user 0.002s sys 0.005s
W20260809 09:34:19.628935 6004 tablet.cc:2369] T e4583229efdb4354b416add4b5272822 P a42f7eccc4104d82ac636aea66ddea99: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 09 09:34:19 dist-test-slave-7kzw krb5kdc[4588](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268058, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.190@KRBTEST.COM for HTTP/127.20.125.148@KRBTEST.COM
I20260809 09:34:19.659734 6030 raft_consensus.cc:482] T e4583229efdb4354b416add4b5272822 P a42f7eccc4104d82ac636aea66ddea99 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 09:34:19.659960 6030 raft_consensus.cc:504] T e4583229efdb4354b416add4b5272822 P a42f7eccc4104d82ac636aea66ddea99 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e61f59be12f343cb8a03d3433b9816d6" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 45849 } } peers { permanent_uuid: "a42f7eccc4104d82ac636aea66ddea99" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 45957 } } peers { permanent_uuid: "fdf79434349b4f6aa8e0cfc7174d0421" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 46169 } }
I20260809 09:34:19.661213 6030 leader_election.cc:288] T e4583229efdb4354b416add4b5272822 P a42f7eccc4104d82ac636aea66ddea99 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e61f59be12f343cb8a03d3433b9816d6 (127.20.125.130:45849), fdf79434349b4f6aa8e0cfc7174d0421 (127.20.125.129:46169)
W20260809 09:34:19.664294 5814 tablet.cc:2369] T e4583229efdb4354b416add4b5272822 P e61f59be12f343cb8a03d3433b9816d6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 09:34:19.670300 6005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:34:19.662186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.129:46169 (local address 127.20.125.131:38005)
0809 09:34:19.662355 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0809 09:34:19.662371 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 09:34:19.662601 (+ 230us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:34:19.663141 (+ 540us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:34:19.663144 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:34:19.663876 (+ 732us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 09:34:19.665333 (+ 1457us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:34:19.665345 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:34:19.667193 (+ 1848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:34:19.667197 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:34:19.668634 (+ 1437us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:34:19.668647 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:34:19.669858 (+ 1211us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:34:19.669862 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:34:19.669970 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:34:19.669995 (+ 25us) client_negotiation.cc:769] Sending connection context
0809 09:34:19.670074 (+ 79us) client_negotiation.cc:240] Negotiation successful
0809 09:34:19.670152 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":73}
I20260809 09:34:19.671042 6019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:34:19.661613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.131:38005 (local address 127.20.125.129:46169)
0809 09:34:19.661926 (+ 313us) server_negotiation.cc:206] Beginning negotiation
0809 09:34:19.661932 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 09:34:19.662511 (+ 579us) server_negotiation.cc:407] Connection header received
0809 09:34:19.662808 (+ 297us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:34:19.662812 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:34:19.662896 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:34:19.663079 (+ 183us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 09:34:19.665905 (+ 2826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:34:19.667024 (+ 1119us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:34:19.668820 (+ 1796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:34:19.669594 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:34:19.669711 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:34:19.669778 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0809 09:34:19.670826 (+ 1048us) server_negotiation.cc:299] Negotiation successful
0809 09:34:19.670914 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":174}
I20260809 09:34:19.671684 5546 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e4583229efdb4354b416add4b5272822" candidate_uuid: "a42f7eccc4104d82ac636aea66ddea99" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fdf79434349b4f6aa8e0cfc7174d0421" is_pre_election: true
I20260809 09:34:19.672049 5546 raft_consensus.cc:2454] T e4583229efdb4354b416add4b5272822 P fdf79434349b4f6aa8e0cfc7174d0421 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a42f7eccc4104d82ac636aea66ddea99 in term 0.
I20260809 09:34:19.673027 5842 leader_election.cc:302] T e4583229efdb4354b416add4b5272822 P a42f7eccc4104d82ac636aea66ddea99 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a42f7eccc4104d82ac636aea66ddea99, fdf79434349b4f6aa8e0cfc7174d0421; no voters:
I20260809 09:34:19.673363 6030 raft_consensus.cc:2781] T e4583229efdb4354b416add4b5272822 P a42f7eccc4104d82ac636aea66ddea99 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 09:34:19.673449 6030 raft_consensus.cc:482] T e4583229efdb4354b416add4b5272822 P a42f7eccc4104d82ac636aea66ddea99 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 09:34:19.673493 6030 raft_consensus.cc:3037] T e4583229efdb4354b416add4b5272822 P a42f7eccc4104d82ac636aea66ddea99 [term 0 FOLLOWER]: Advancing to term 1
I20260809 09:34:19.674580 6020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:34:19.663552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.131:51247 (local address 127.20.125.130:45849)
0809 09:34:19.663711 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0809 09:34:19.663715 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 09:34:19.669624 (+ 5909us) server_negotiation.cc:407] Connection header received
0809 09:34:19.669839 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:34:19.669842 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:34:19.669909 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:34:19.669960 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 09:34:19.670602 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:34:19.671394 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:34:19.673059 (+ 1665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:34:19.673747 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:34:19.673850 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:34:19.673902 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0809 09:34:19.674326 (+ 424us) server_negotiation.cc:299] Negotiation successful
0809 09:34:19.674414 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 09:34:19.675000 6034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:34:19.665167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.130:45849 (local address 127.20.125.131:51247)
0809 09:34:19.669508 (+ 4341us) negotiation.cc:107] Waiting for socket to connect
0809 09:34:19.669524 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 09:34:19.669705 (+ 181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:34:19.670114 (+ 409us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:34:19.670117 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:34:19.670136 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 09:34:19.670409 (+ 273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:34:19.670416 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:34:19.671558 (+ 1142us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:34:19.671561 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:34:19.672852 (+ 1291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:34:19.672870 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:34:19.674051 (+ 1181us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:34:19.674055 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:34:19.674167 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:34:19.674193 (+ 26us) client_negotiation.cc:769] Sending connection context
0809 09:34:19.674723 (+ 530us) client_negotiation.cc:240] Negotiation successful
0809 09:34:19.674812 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4260,"thread_start_us":1042,"threads_started":1}
I20260809 09:34:19.675669 6030 raft_consensus.cc:504] T e4583229efdb4354b416add4b5272822 P a42f7eccc4104d82ac636aea66ddea99 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e61f59be12f343cb8a03d3433b9816d6" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 45849 } } peers { permanent_uuid: "a42f7eccc4104d82ac636aea66ddea99" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 45957 } } peers { permanent_uuid: "fdf79434349b4f6aa8e0cfc7174d0421" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 46169 } }
I20260809 09:34:19.676077 6030 leader_election.cc:288] T e4583229efdb4354b416add4b5272822 P a42f7eccc4104d82ac636aea66ddea99 [CANDIDATE]: Term 1 election: Requested vote from peers e61f59be12f343cb8a03d3433b9816d6 (127.20.125.130:45849), fdf79434349b4f6aa8e0cfc7174d0421 (127.20.125.129:46169)
I20260809 09:34:19.676884 5546 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e4583229efdb4354b416add4b5272822" candidate_uuid: "a42f7eccc4104d82ac636aea66ddea99" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fdf79434349b4f6aa8e0cfc7174d0421"
I20260809 09:34:19.677023 5546 raft_consensus.cc:3037] T e4583229efdb4354b416add4b5272822 P fdf79434349b4f6aa8e0cfc7174d0421 [term 0 FOLLOWER]: Advancing to term 1
I20260809 09:34:19.677232 5745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e4583229efdb4354b416add4b5272822" candidate_uuid: "a42f7eccc4104d82ac636aea66ddea99" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e61f59be12f343cb8a03d3433b9816d6"
I20260809 09:34:19.677480 5745 raft_consensus.cc:3037] T e4583229efdb4354b416add4b5272822 P e61f59be12f343cb8a03d3433b9816d6 [term 0 FOLLOWER]: Advancing to term 1
I20260809 09:34:19.678412 5546 raft_consensus.cc:2454] T e4583229efdb4354b416add4b5272822 P fdf79434349b4f6aa8e0cfc7174d0421 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a42f7eccc4104d82ac636aea66ddea99 in term 1.
I20260809 09:34:19.677237 5749 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e4583229efdb4354b416add4b5272822" candidate_uuid: "a42f7eccc4104d82ac636aea66ddea99" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e61f59be12f343cb8a03d3433b9816d6" is_pre_election: true
I20260809 09:34:19.678922 5842 leader_election.cc:302] T e4583229efdb4354b416add4b5272822 P a42f7eccc4104d82ac636aea66ddea99 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a42f7eccc4104d82ac636aea66ddea99, fdf79434349b4f6aa8e0cfc7174d0421; no voters:
I20260809 09:34:19.679157 6030 raft_consensus.cc:2781] T e4583229efdb4354b416add4b5272822 P a42f7eccc4104d82ac636aea66ddea99 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 09:34:19.679400 6030 raft_consensus.cc:686] T e4583229efdb4354b416add4b5272822 P a42f7eccc4104d82ac636aea66ddea99 [term 1 LEADER]: Becoming Leader. State: Replica: a42f7eccc4104d82ac636aea66ddea99, State: Running, Role: LEADER
I20260809 09:34:19.679811 5745 raft_consensus.cc:2454] T e4583229efdb4354b416add4b5272822 P e61f59be12f343cb8a03d3433b9816d6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a42f7eccc4104d82ac636aea66ddea99 in term 1.
I20260809 09:34:19.679770 6030 consensus_queue.cc:237] T e4583229efdb4354b416add4b5272822 P a42f7eccc4104d82ac636aea66ddea99 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e61f59be12f343cb8a03d3433b9816d6" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 45849 } } peers { permanent_uuid: "a42f7eccc4104d82ac636aea66ddea99" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 45957 } } peers { permanent_uuid: "fdf79434349b4f6aa8e0cfc7174d0421" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 46169 } }
I20260809 09:34:19.684966 5292 catalog_manager.cc:5495] T e4583229efdb4354b416add4b5272822 P a42f7eccc4104d82ac636aea66ddea99 reported cstate change: term changed from 0 to 1, leader changed from <none> to a42f7eccc4104d82ac636aea66ddea99 (127.20.125.131). New cstate: current_term: 1 leader_uuid: "a42f7eccc4104d82ac636aea66ddea99" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e61f59be12f343cb8a03d3433b9816d6" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 45849 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a42f7eccc4104d82ac636aea66ddea99" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 45957 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fdf79434349b4f6aa8e0cfc7174d0421" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 46169 } health_report { overall_health: UNKNOWN } } }
I20260809 09:34:19.714442 6023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:34:19.709488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50002 (local address 127.20.125.131:45957)
0809 09:34:19.709645 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0809 09:34:19.709650 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 09:34:19.709737 (+ 87us) server_negotiation.cc:407] Connection header received
0809 09:34:19.709972 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:34:19.709977 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:34:19.710070 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:34:19.710185 (+ 115us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 09:34:19.710443 (+ 258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:34:19.711231 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:34:19.712424 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:34:19.712669 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:34:19.712790 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:34:19.713111 (+ 321us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 09:34:19.713798 (+ 687us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 09:34:19.713918 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0809 09:34:19.714241 (+ 323us) server_negotiation.cc:299] Negotiation successful
0809 09:34:19.714301 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260809 09:34:19.728884 5546 raft_consensus.cc:1261] T e4583229efdb4354b416add4b5272822 P fdf79434349b4f6aa8e0cfc7174d0421 [term 1 FOLLOWER]: Refusing update from remote peer a42f7eccc4104d82ac636aea66ddea99: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 09:34:19.729825 6037 consensus_queue.cc:1035] T e4583229efdb4354b416add4b5272822 P a42f7eccc4104d82ac636aea66ddea99 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fdf79434349b4f6aa8e0cfc7174d0421" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 46169 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 09:34:19.744058 5745 raft_consensus.cc:1261] T e4583229efdb4354b416add4b5272822 P e61f59be12f343cb8a03d3433b9816d6 [term 1 FOLLOWER]: Refusing update from remote peer a42f7eccc4104d82ac636aea66ddea99: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 09:34:19.746673 6037 consensus_queue.cc:1035] T e4583229efdb4354b416add4b5272822 P a42f7eccc4104d82ac636aea66ddea99 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e61f59be12f343cb8a03d3433b9816d6" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 45849 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 09:34:19.759338 6047 mvcc.cc:205] Tried to move back new op lower bound from 7316553972638384128 to 7316553972457787392. Current Snapshot: MvccSnapshot[applied={T|T < 7316553972638384128}]
I20260809 09:34:19.770395 6044 mvcc.cc:205] Tried to move back new op lower bound from 7316553972638384128 to 7316553972457787392. Current Snapshot: MvccSnapshot[applied={T|T < 7316553972638384128 or (T in {7316553972638384128})}]
I20260809 09:34:19.774008 5292 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60318:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 09:34:19.774397 5292 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60318:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 09:34:19.778084 5292 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 04886f1a37334005842e45ca4685463b: Sending DeleteTablet for 3 replicas of tablet e4583229efdb4354b416add4b5272822
I20260809 09:34:19.779467 5516 tablet_service.cc:1514] Processing DeleteTablet for tablet e4583229efdb4354b416add4b5272822 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 09:34:19 UTC) from {username='kudu', principal='kudu/127.20.125.190@KRBTEST.COM'} at 127.0.0.1:49912
I20260809 09:34:19.780812 5908 tablet_service.cc:1514] Processing DeleteTablet for tablet e4583229efdb4354b416add4b5272822 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 09:34:19 UTC) from {username='kudu', principal='kudu/127.20.125.190@KRBTEST.COM'} at 127.0.0.1:49990
I20260809 09:34:19.782861 20982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu with pid 5419
I20260809 09:34:19.783476 5712 tablet_service.cc:1514] Processing DeleteTablet for tablet e4583229efdb4354b416add4b5272822 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 09:34:19 UTC) from {username='kudu', principal='kudu/127.20.125.190@KRBTEST.COM'} at 127.0.0.1:41386
I20260809 09:34:19.793452 6061 tablet_replica.cc:330] T e4583229efdb4354b416add4b5272822 P e61f59be12f343cb8a03d3433b9816d6: stopping tablet replica
I20260809 09:34:19.793891 6061 raft_consensus.cc:2229] T e4583229efdb4354b416add4b5272822 P e61f59be12f343cb8a03d3433b9816d6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 09:34:19.794322 6061 raft_consensus.cc:2258] T e4583229efdb4354b416add4b5272822 P e61f59be12f343cb8a03d3433b9816d6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 09:34:19.797333 6060 tablet_replica.cc:330] T e4583229efdb4354b416add4b5272822 P a42f7eccc4104d82ac636aea66ddea99: stopping tablet replica
I20260809 09:34:19.797721 6060 raft_consensus.cc:2229] T e4583229efdb4354b416add4b5272822 P a42f7eccc4104d82ac636aea66ddea99 [term 1 LEADER]: Raft consensus shutting down.
I20260809 09:34:19.798122 6061 ts_tablet_manager.cc:1905] T e4583229efdb4354b416add4b5272822 P e61f59be12f343cb8a03d3433b9816d6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 09:34:19.798656 6060 raft_consensus.cc:2258] T e4583229efdb4354b416add4b5272822 P a42f7eccc4104d82ac636aea66ddea99 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 09:34:19.802927 6061 ts_tablet_manager.cc:1918] T e4583229efdb4354b416add4b5272822 P e61f59be12f343cb8a03d3433b9816d6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 09:34:19.803046 6061 log.cc:1196] T e4583229efdb4354b416add4b5272822 P e61f59be12f343cb8a03d3433b9816d6: Deleting WAL directory at /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786267941944493-20982-0/minicluster-data/ts-1/wal/wals/e4583229efdb4354b416add4b5272822
I20260809 09:34:19.803401 6061 ts_tablet_manager.cc:1939] T e4583229efdb4354b416add4b5272822 P e61f59be12f343cb8a03d3433b9816d6: Deleting consensus metadata
I20260809 09:34:19.805258 6060 ts_tablet_manager.cc:1905] T e4583229efdb4354b416add4b5272822 P a42f7eccc4104d82ac636aea66ddea99: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 09:34:19.810184 5271 catalog_manager.cc:4841] TS e61f59be12f343cb8a03d3433b9816d6 (127.20.125.130:45849): tablet e4583229efdb4354b416add4b5272822 (table test-table [id=555201544d5e426cadcac6ae958ad460]) successfully deleted
I20260809 09:34:19.810305 6060 ts_tablet_manager.cc:1918] T e4583229efdb4354b416add4b5272822 P a42f7eccc4104d82ac636aea66ddea99: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 09:34:19.810415 6060 log.cc:1196] T e4583229efdb4354b416add4b5272822 P a42f7eccc4104d82ac636aea66ddea99: Deleting WAL directory at /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786267941944493-20982-0/minicluster-data/ts-2/wal/wals/e4583229efdb4354b416add4b5272822
I20260809 09:34:19.810818 6060 ts_tablet_manager.cc:1939] T e4583229efdb4354b416add4b5272822 P a42f7eccc4104d82ac636aea66ddea99: Deleting consensus metadata
I20260809 09:34:19.812176 5270 catalog_manager.cc:4841] TS a42f7eccc4104d82ac636aea66ddea99 (127.20.125.131:45957): tablet e4583229efdb4354b416add4b5272822 (table test-table [id=555201544d5e426cadcac6ae958ad460]) successfully deleted
W20260809 09:34:19.815179 5266 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.20.125.129:46169 (error 108)
I20260809 09:34:19.816792 20982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu with pid 5627
I20260809 09:34:19.816850 6018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:34:19.816507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.129:46169 (local address 127.0.0.1:49918)
0809 09:34:19.816668 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0809 09:34:19.816756 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.129:46169: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260809 09:34:19.828774 5266 catalog_manager.cc:4584] TS fdf79434349b4f6aa8e0cfc7174d0421 (127.20.125.129:46169): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e4583229efdb4354b416add4b5272822: Network error: Client connection negotiation failed: client connection to 127.20.125.129:46169: connect: Connection refused (error 111)
I20260809 09:34:19.866700 20982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu with pid 5820
I20260809 09:34:19.899891 6018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:34:19.881903 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.129:46169 (local address 127.0.0.1:49930)
0809 09:34:19.899656 (+ 17753us) negotiation.cc:107] Waiting for socket to connect
0809 09:34:19.899763 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.129:46169: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":17606}
I20260809 09:34:19.906824 6018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:34:19.903091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.129:46169 (local address 127.0.0.1:49936)
0809 09:34:19.906616 (+ 3525us) negotiation.cc:107] Waiting for socket to connect
0809 09:34:19.906706 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.129:46169: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":3405}
I20260809 09:34:19.913280 20982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu with pid 5246
I20260809 09:34:19.927767 20982 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260809 09:34:20.275326 20982 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260809 09:34:20.275463 20982 mini_ranger.cc:66] Stopping Ranger...
I20260809 09:34:20.632606 20982 mini_ranger.cc:68] Stopped Ranger
2026-08-09 09:34:20.632 UTC [4627] LOG: received smart shutdown request
2026-08-09 09:34:20.637 UTC [4627] LOG: background worker "logical replication launcher" (PID 4639) exited with exit code 1
2026-08-09 09:34:20.637 UTC [4631] LOG: shutting down
2026-08-09 09:34:20.802 UTC [4627] LOG: database system is shut down
2026-08-09T09:34:20Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (50401 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786267941944493-20982-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 09:34:20 dist-test-slave-7kzw krb5kdc[6076](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 09:34:20 dist-test-slave-7kzw krb5kdc[6076](info): set up 2 sockets
Aug 09 09:34:20 dist-test-slave-7kzw krb5kdc[6076](info): commencing operation
krb5kdc: starting...
W20260809 09:34:22.179198 20982 mini_kdc.cc:121] Time spent starting KDC: real 1.232s user 0.001s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786267941944493-20982-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786267941944493-20982-0/krb5kdc/test-user.keytab.
Aug 09 09:34:22 dist-test-slave-7kzw krb5kdc[6076](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268062, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T09:34:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T09:34:22Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.20.125.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.20.125.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.20.125.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786267941944493-20982-0/krb5kdc/rangeradmin_127.20.125.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.20.125.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786267941944493-20982-0/krb5kdc/rangeradmin_127.20.125.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.20.125.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.20.125.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.20.125.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786267941944493-20982-0/krb5kdc/rangerlookup_127.20.125.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.20.125.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786267941944493-20982-0/krb5kdc/rangerlookup_127.20.125.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.20.125.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.20.125.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.20.125.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786267941944493-20982-0/krb5kdc/HTTP_127.20.125.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.20.125.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786267941944493-20982-0/krb5kdc/HTTP_127.20.125.148@KRBTEST.COM.keytab.
I20260809 09:34:22.364810 20982 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786267941944493-20982-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskYHpNFG/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786267941944493-20982-0/minicluster-data/postgres -l logfile start
2026-08-09 09:34:25.085 UTC [6120] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-09 09:34:25.085 UTC [6120] LOG: listening on IPv4 address "127.20.125.148", port 34533
2026-08-09 09:34:25.088 UTC [6120] LOG: listening on Unix socket "/tmp/.s.PGSQL.34533"
2026-08-09 09:34:25.111 UTC [6125] LOG: database system was shut down at 2026-08-09 09:34:23 UTC
2026-08-09 09:34:25.115 UTC [6120] LOG: database system is ready to accept connections
I20260809 09:34:26.295609 20982 mini_postgres.cc:96] Postgres bound to 34533
2026-08-09 09:34:26.303 UTC [6134] FATAL: database "slave" does not exist
127.20.125.148:34533 - accepting connections
I20260809 09:34:26.303931 20982 mini_ranger.cc:161] Starting Ranger...
I20260809 09:34:26.320302 20982 mini_ranger.cc:84] Created miniranger Postgres user
I20260809 09:34:26.825460 20982 mini_ranger.cc:87] Created ranger Postgres database
I20260809 09:34:26.825563 20982 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786267941944493-20982-0/minicluster-data/ranger-admin
2026-08-09 09:34:27,129 [I] DB FLAVOR :POSTGRES
2026-08-09 09:34:27,130 [I] --------- Verifying Ranger DB connection ---------
2026-08-09 09:34:27,130 [I] Checking connection..
2026-08-09 09:34:27,130 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYHpNFG/build/debug/bin/postgresql.jar:/tmp/dist-test-taskYHpNFG/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.125.148:34533/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-09 09:34:27,386 [I] Checking connection passed.
2026-08-09 09:34:27,386 [I] --------- Verifying version history table ---------
2026-08-09 09:34:27,386 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYHpNFG/build/debug/bin/postgresql.jar:/tmp/dist-test-taskYHpNFG/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.125.148:34533/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 09:34:27,627 [I] Table x_db_version_h does not exist in database ranger
2026-08-09 09:34:27,627 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYHpNFG/build/debug/bin/postgresql.jar:/tmp/dist-test-taskYHpNFG/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.125.148:34533/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 09:34:27,892 [I] Table x_db_version_h does not exist in database ranger
2026-08-09 09:34:27,892 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-09 09:34:27,892 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYHpNFG/build/debug/bin/postgresql.jar:/tmp/dist-test-taskYHpNFG/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.125.148:34533/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskYHpNFG/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-09 09:34:28.155 UTC [6234] WARNING: there is no transaction in progress
2026-08-09 09:34:28,162 [I] create_dbversion_catalog.sql file imported successfully
2026-08-09 09:34:28,162 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYHpNFG/build/debug/bin/postgresql.jar:/tmp/dist-test-taskYHpNFG/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.125.148:34533/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 09:34:28,434 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-09 09:34:28,435 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-09 09:34:28,435 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYHpNFG/build/debug/bin/postgresql.jar:/tmp/dist-test-taskYHpNFG/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.125.148:34533/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-09 09:34:28,702 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYHpNFG/build/debug/bin/postgresql.jar:/tmp/dist-test-taskYHpNFG/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.125.148:34533/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-09 09:34:28,958 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYHpNFG/build/debug/bin/postgresql.jar:/tmp/dist-test-taskYHpNFG/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.125.148:34533/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-7kzw.c.gcp-upstream.internal','N') ;"
2026-08-09 09:34:29,246 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYHpNFG/build/debug/bin/postgresql.jar:/tmp/dist-test-taskYHpNFG/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.125.148:34533/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-09 09:34:29,525 [I] Table x_portal_user does not exist in database ranger
2026-08-09 09:34:29,525 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYHpNFG/build/debug/bin/postgresql.jar:/tmp/dist-test-taskYHpNFG/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.125.148:34533/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-09 09:34:29,790 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-09 09:34:29,791 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYHpNFG/build/debug/bin/postgresql.jar:/tmp/dist-test-taskYHpNFG/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.125.148:34533/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-09 09:34:30,069 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYHpNFG/build/debug/bin/postgresql.jar:/tmp/dist-test-taskYHpNFG/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.125.148:34533/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-09 09:34:30,338 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-09 09:34:30,338 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYHpNFG/build/debug/bin/postgresql.jar:/tmp/dist-test-taskYHpNFG/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.125.148:34533/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskYHpNFG/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-09 09:34:31.350 UTC [6384] WARNING: there is no transaction in progress
2026-08-09 09:34:31.366 UTC [6384] WARNING: there is no transaction in progress
2026-08-09 09:34:31.384 UTC [6384] WARNING: there is no transaction in progress
2026-08-09 09:34:31.399 UTC [6384] WARNING: there is no transaction in progress
2026-08-09 09:34:31.416 UTC [6384] WARNING: there is no transaction in progress
2026-08-09 09:34:31.436 UTC [6384] WARNING: there is no transaction in progress
2026-08-09 09:34:31.492 UTC [6384] WARNING: there is no transaction in progress
2026-08-09 09:34:31.499 UTC [6384] WARNING: there is no transaction in progress
2026-08-09 09:34:31.513 UTC [6384] WARNING: there is no transaction in progress
2026-08-09 09:34:31.524 UTC [6384] WARNING: there is no transaction in progress
2026-08-09 09:34:31.535 UTC [6384] WARNING: there is no transaction in progress
2026-08-09 09:34:31.557 UTC [6384] WARNING: there is no transaction in progress
2026-08-09 09:34:31.567 UTC [6384] WARNING: there is no transaction in progress
2026-08-09 09:34:31.574 UTC [6384] WARNING: there is no transaction in progress
2026-08-09 09:34:31.581 UTC [6384] WARNING: there is no transaction in progress
2026-08-09 09:34:31.588 UTC [6384] WARNING: there is no transaction in progress
2026-08-09 09:34:32,246 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-09 09:34:32,247 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYHpNFG/build/debug/bin/postgresql.jar:/tmp/dist-test-taskYHpNFG/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.125.148:34533/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-09 09:34:32,494 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-09 09:34:32,495 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYHpNFG/build/debug/bin/postgresql.jar:/tmp/dist-test-taskYHpNFG/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.125.148:34533/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-09 09:34:32,762 [I] Table x_portal_user already exists in database 'ranger'
2026-08-09 09:34:32,763 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYHpNFG/build/debug/bin/postgresql.jar:/tmp/dist-test-taskYHpNFG/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.125.148:34533/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-09 09:34:33,027 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-09 09:34:33,027 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYHpNFG/build/debug/bin/postgresql.jar:/tmp/dist-test-taskYHpNFG/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.125.148:34533/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-09 09:34:33,280 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYHpNFG/build/debug/bin/postgresql.jar:/tmp/dist-test-taskYHpNFG/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.125.148:34533/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-09 09:34:33,549 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYHpNFG/build/debug/bin/postgresql.jar:/tmp/dist-test-taskYHpNFG/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.125.148:34533/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-7kzw.c.gcp-upstream.internal';"
2026-08-09 09:34:33,803 [I] CORE_DB_SCHEMA import status has been updated
2026-08-09 09:34:33,804 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYHpNFG/build/debug/bin/postgresql.jar:/tmp/dist-test-taskYHpNFG/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.125.148:34533/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-09 09:34:34,065 [I] DB_PATCHES have already been applied
I20260809 09:34:34.070976 20982 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786267941944493-20982-0/minicluster-data/ranger-admin:/tmp/dist-test-taskYHpNFG/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskYHpNFG/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskYHpNFG/build/debug/bin/postgresql.jar:/tmp/dist-test-taskYHpNFG/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260809 09:34:34.071044 20982 mini_ranger.cc:193] Using host: 127.20.125.148
I20260809 09:34:34.075623 20982 mini_ranger.cc:227] Ranger admin URL: http://127.20.125.148:58915
Aug 09, 2026 9:34:34 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 09, 2026 9:34:34 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 09, 2026 9:34:34 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786267941944493-20982-0/minicluster-data/ranger-admin/ews/webapp
Aug 09, 2026 9:34:34 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786267941944493-20982-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 09, 2026 9:34:34 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786267941944493-20982-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 09, 2026 9:34:34 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786267941944493-20982-0/minicluster-data/ranger-admin/ews/webapp].
Aug 09, 2026 9:34:34 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260809 09:34:35.513582 20982 mini_ranger.cc:160] Time spent starting Ranger: real 9.210s user 0.007s sys 0.005s
Aug 09, 2026 9:34:55 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 09, 2026 9:34:55 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 09, 2026 9:34:55 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 09, 2026 9:34:55 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 09, 2026 9:34:55 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 09, 2026 9:34:55 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 09, 2026 9:34:55 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 09, 2026 9:34:55 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 09, 2026 9:34:55 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 09, 2026 9:34:55 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 09, 2026 9:34:55 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 09, 2026 9:34:55 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 09, 2026 9:34:55 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 09, 2026 9:34:55 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 09, 2026 9:34:55 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 09, 2026 9:34:55 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 09, 2026 9:34:55 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 09, 2026 9:34:56 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260809 09:34:56.971282 20982 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.20.125.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.20.125.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.20.125.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786267941944493-20982-0/krb5kdc/rangerkms_127.20.125.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.20.125.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786267941944493-20982-0/krb5kdc/rangerkms_127.20.125.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.20.125.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.20.125.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.20.125.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786267941944493-20982-0/krb5kdc/HTTP_127.20.125.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.20.125.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786267941944493-20982-0/krb5kdc/HTTP_127.20.125.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 09 09:34:57 dist-test-slave-7kzw krb5kdc[6076](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268097, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260809 09:34:57.077399 20982 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260809 09:34:57.088037 20982 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260809 09:34:58.058609 20982 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260809 09:34:58.058706 20982 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786267941944493-20982-0/minicluster-data/ranger-kms
I20260809 09:34:58.058727 20982 mini_ranger_kms.cc:227] Using postgres at 127.20.125.148:34533
2026-08-09 09:34:58,261 [I] DB FLAVOR :POSTGRES
2026-08-09 09:34:58,261 [I] --------- Verifying Ranger DB connection ---------
2026-08-09 09:34:58,261 [I] Checking connection
2026-08-09 09:34:58,261 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYHpNFG/build/debug/bin/postgresql.jar:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786267941944493-20982-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.125.148:34533/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-09 09:34:58,503 [I] connection success
2026-08-09 09:34:58,503 [I] --------- Verifying Ranger DB tables ---------
2026-08-09 09:34:58,503 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-09 09:34:58,503 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYHpNFG/build/debug/bin/postgresql.jar:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786267941944493-20982-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.125.148:34533/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-09 09:34:58,765 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-09 09:34:58,766 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-09 09:34:58,766 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-09 09:34:58,766 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYHpNFG/build/debug/bin/postgresql.jar:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786267941944493-20982-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.125.148:34533/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786267941944493-20982-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-09 09:34:59,062 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260809 09:34:59.515625 20982 mini_ranger_kms.cc:310] Created kms service
I20260809 09:34:59.726804 20982 mini_ranger_kms.cc:326] Created kudu user
I20260809 09:34:59.779258 20982 mini_ranger_kms.cc:343] Created rangerkms user
Aug 09 09:34:59 dist-test-slave-7kzw krb5kdc[6076](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268097, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.20.125.148@KRBTEST.COM
I20260809 09:35:00.249423 20982 mini_ranger_kms.cc:384] Added ranger policy
I20260809 09:35:00.249600 20982 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786267941944493-20982-0/minicluster-data/ranger-kms:/tmp/dist-test-taskYHpNFG/build/debug/bin/postgresql.jar:/tmp/dist-test-taskYHpNFG/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskYHpNFG/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260809 09:35:00.249624 20982 mini_ranger_kms.cc:240] Using host: 127.20.125.148
I20260809 09:35:00.254805 20982 mini_ranger_kms.cc:276] Ranger KMS PID: 6635
I20260809 09:35:00.254902 20982 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.20.125.148:57127
Aug 09, 2026 9:35:01 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 09, 2026 9:35:01 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 09, 2026 9:35:01 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786267941944493-20982-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 09, 2026 9:35:01 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786267941944493-20982-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 09, 2026 9:35:02 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786267941944493-20982-0/minicluster-data/ranger-kms/ews/webapp].
Aug 09, 2026 9:35:02 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 09 09:35:02 dist-test-slave-7kzw krb5kdc[6076](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786268102, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.20.125.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 09:35:03.082768 20982 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.005s user 0.018s sys 0.169s
I20260809 09:35:03.082912 20982 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260809 09:35:03.083002 20982 mini_ranger_kms.cc:401] 127.20.125.148:57127/kms/v1/keys
Aug 09 09:35:03 dist-test-slave-7kzw krb5kdc[6076](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268102, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.20.125.148@KRBTEST.COM for HTTP/127.20.125.148@KRBTEST.COM
Aug 09 09:35:03 dist-test-slave-7kzw krb5kdc[6076](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268102, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.20.125.148@KRBTEST.COM for HTTP/127.20.125.148@KRBTEST.COM
Aug 09 09:35:06 dist-test-slave-7kzw krb5kdc[6076](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268106, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.20.125.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.125.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.125.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.190.keytab.
Entry for principal kudu/127.20.125.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.190.keytab.
I20260809 09:35:06.297689 20982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
/tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786267941944493-20982-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786267941944493-20982-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786267941944493-20982-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786267941944493-20982-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.125.190:42791
--webserver_interface=127.20.125.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.190.keytab
--principal=kudu/127.20.125.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.125.148:33717
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.125.190:42791
--ranger_config_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786267941944493-20982-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.20.125.148:57127/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786267941944493-20982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786267941944493-20982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786267941944493-20982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 09:35:06.423156 6706 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 09:35:06.423470 6706 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 09:35:06.423516 6706 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 09:35:06.428174 6706 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 09:35:06.428241 6706 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 09:35:06.428263 6706 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 09:35:06.428282 6706 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 09:35:06.428299 6706 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 09:35:06.434455 6706 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.125.148:33717
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786267941944493-20982-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786267941944493-20982-0/minicluster-data/master-0/wal
--ranger_kms_url=127.20.125.148:57127/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.20.125.190:42791
--ranger_config_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786267941944493-20982-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.190.keytab
--principal=kudu/127.20.125.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.125.190:42791
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786267941944493-20982-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.125.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6706
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786267941944493-20982-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 08:38:55 UTC on ccycloud.quasar-njjleu.root.comops.site
build id 4575935
I20260809 09:35:06.435698 6706 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 09:35:06.436793 6706 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 09:35:06.446202 6719 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:35:06.446239 6713 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:35:06.446218 6716 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:35:06.446552 6706 server_base.cc:1034] running on GCE node
I20260809 09:35:06.447382 6706 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 09:35:06.448559 6706 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 09:35:06.449775 6706 hybrid_clock.cc:648] HybridClock initialized: now 1786268106449745 us; error 42 us; skew 500 ppm
Aug 09 09:35:06 dist-test-slave-7kzw krb5kdc[6076](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268106, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 09:35:06.456986 6706 init.cc:377] Logged in from keytab as kudu/127.20.125.190@KRBTEST.COM (short username kudu)
I20260809 09:35:06.457949 6706 webserver.cc:460] Webserver started at http://127.20.125.190:36387/ using document root <none> and password file <none>
I20260809 09:35:06.458585 6706 fs_manager.cc:359] Metadata directory not provided
I20260809 09:35:06.458671 6706 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 09:35:06.458946 6706 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 09:35:06 dist-test-slave-7kzw krb5kdc[6076](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268106, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.190@KRBTEST.COM for HTTP/127.20.125.148@KRBTEST.COM
I20260809 09:35:06.523864 6706 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786267941944493-20982-0/minicluster-data/master-0/data/instance:
uuid: "5c09265aec1a48b894b76b40aa1fc8b7"
format_stamp: "Formatted at 2026-08-09 09:35:06 on dist-test-slave-7kzw"
server_key: "7aade926d57b40ca3f567b26b42433e3"
server_key_iv: "469bd569b441fbe6a632f5c33fb617c6"
server_key_version: "kuduclusterkey@0"
I20260809 09:35:06.524762 6706 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786267941944493-20982-0/minicluster-data/master-0/wal/instance:
uuid: "5c09265aec1a48b894b76b40aa1fc8b7"
format_stamp: "Formatted at 2026-08-09 09:35:06 on dist-test-slave-7kzw"
server_key: "7aade926d57b40ca3f567b26b42433e3"
server_key_iv: "469bd569b441fbe6a632f5c33fb617c6"
server_key_version: "kuduclusterkey@0"
I20260809 09:35:06.530409 6706 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 09:35:06.550163 6735 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 09:35:06.551754 6706 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 09:35:06.551928 6706 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786267941944493-20982-0/minicluster-data/master-0/data,/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786267941944493-20982-0/minicluster-data/master-0/wal
uuid: "5c09265aec1a48b894b76b40aa1fc8b7"
format_stamp: "Formatted at 2026-08-09 09:35:06 on dist-test-slave-7kzw"
server_key: "7aade926d57b40ca3f567b26b42433e3"
server_key_iv: "469bd569b441fbe6a632f5c33fb617c6"
server_key_version: "kuduclusterkey@0"
I20260809 09:35:06.552093 6706 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786267941944493-20982-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786267941944493-20982-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786267941944493-20982-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 09:35:06.590210 6706 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 09:35:06.593730 6706 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 09:35:06.593964 6706 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 09:35:06.602066 6706 rpc_server.cc:307] RPC server started. Bound to: 127.20.125.190:42791
I20260809 09:35:06.602087 6808 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.125.190:42791 every 8 connection(s)
I20260809 09:35:06.603183 6706 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786267941944493-20982-0/minicluster-data/master-0/data/info.pb
I20260809 09:35:06.606115 6810 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 09:35:06.608584 20982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu as pid 6706
I20260809 09:35:06.608752 20982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786267941944493-20982-0/minicluster-data/master-0/wal/instance
I20260809 09:35:06.609059 20982 external_mini_cluster.cc:1428] Setting key 5087c30cff516ae0157c510c9e0e19c9
I20260809 09:35:06.612246 6810 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5c09265aec1a48b894b76b40aa1fc8b7: Bootstrap starting.
I20260809 09:35:06.614871 6810 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5c09265aec1a48b894b76b40aa1fc8b7: Neither blocks nor log segments found. Creating new log.
I20260809 09:35:06.615732 6810 log.cc:824] T 00000000000000000000000000000000 P 5c09265aec1a48b894b76b40aa1fc8b7: Log is configured to *not* fsync() on all Append() calls
I20260809 09:35:06.618191 6810 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5c09265aec1a48b894b76b40aa1fc8b7: No bootstrap required, opened a new log
Aug 09 09:35:06 dist-test-slave-7kzw krb5kdc[6076](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268106, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.125.190@KRBTEST.COM
I20260809 09:35:06.621883 6810 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5c09265aec1a48b894b76b40aa1fc8b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c09265aec1a48b894b76b40aa1fc8b7" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 42791 } }
I20260809 09:35:06.622125 6810 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5c09265aec1a48b894b76b40aa1fc8b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 09:35:06.622191 6810 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5c09265aec1a48b894b76b40aa1fc8b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c09265aec1a48b894b76b40aa1fc8b7, State: Initialized, Role: FOLLOWER
I20260809 09:35:06.622704 6810 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5c09265aec1a48b894b76b40aa1fc8b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c09265aec1a48b894b76b40aa1fc8b7" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 42791 } }
I20260809 09:35:06.622887 6810 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5c09265aec1a48b894b76b40aa1fc8b7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 09:35:06.622999 6810 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5c09265aec1a48b894b76b40aa1fc8b7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 09:35:06.623121 6810 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5c09265aec1a48b894b76b40aa1fc8b7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 09:35:06.624068 6810 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5c09265aec1a48b894b76b40aa1fc8b7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c09265aec1a48b894b76b40aa1fc8b7" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 42791 } }
I20260809 09:35:06.624197 6813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:35:06.610523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53530 (local address 127.20.125.190:42791)
0809 09:35:06.610937 (+ 414us) server_negotiation.cc:206] Beginning negotiation
0809 09:35:06.610957 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0809 09:35:06.611000 (+ 43us) server_negotiation.cc:407] Connection header received
0809 09:35:06.611629 (+ 629us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:35:06.611645 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:35:06.611964 (+ 319us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:35:06.612253 (+ 289us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:35:06.613231 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:35:06.614214 (+ 983us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:35:06.615439 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:35:06.615973 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:35:06.616109 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:35:06.619190 (+ 3081us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:35:06.619195 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:35:06.619208 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:35:06.619235 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:35:06.620911 (+ 1676us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:35:06.621388 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:35:06.621395 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:35:06.621409 (+ 14us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:35:06.621520 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:35:06.621864 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:35:06.621867 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:35:06.621869 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:35:06.622408 (+ 539us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:35:06.622654 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0809 09:35:06.622960 (+ 306us) server_negotiation.cc:299] Negotiation successful
0809 09:35:06.623447 (+ 487us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":264,"thread_start_us":127,"threads_started":1}
I20260809 09:35:06.624454 6810 leader_election.cc:302] T 00000000000000000000000000000000 P 5c09265aec1a48b894b76b40aa1fc8b7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5c09265aec1a48b894b76b40aa1fc8b7; no voters:
I20260809 09:35:06.624789 6810 leader_election.cc:288] T 00000000000000000000000000000000 P 5c09265aec1a48b894b76b40aa1fc8b7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 09:35:06.624897 6816 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5c09265aec1a48b894b76b40aa1fc8b7 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 09:35:06.625118 6816 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5c09265aec1a48b894b76b40aa1fc8b7 [term 1 LEADER]: Becoming Leader. State: Replica: 5c09265aec1a48b894b76b40aa1fc8b7, State: Running, Role: LEADER
I20260809 09:35:06.625479 6816 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5c09265aec1a48b894b76b40aa1fc8b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c09265aec1a48b894b76b40aa1fc8b7" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 42791 } }
I20260809 09:35:06.626098 6810 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5c09265aec1a48b894b76b40aa1fc8b7 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 09:35:06.627249 6823 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5c09265aec1a48b894b76b40aa1fc8b7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5c09265aec1a48b894b76b40aa1fc8b7. Latest consensus state: current_term: 1 leader_uuid: "5c09265aec1a48b894b76b40aa1fc8b7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c09265aec1a48b894b76b40aa1fc8b7" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 42791 } } }
I20260809 09:35:06.627254 6820 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5c09265aec1a48b894b76b40aa1fc8b7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5c09265aec1a48b894b76b40aa1fc8b7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c09265aec1a48b894b76b40aa1fc8b7" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 42791 } } }
I20260809 09:35:06.627414 6820 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5c09265aec1a48b894b76b40aa1fc8b7 [sys.catalog]: This master's current role is: LEADER
I20260809 09:35:06.627414 6823 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5c09265aec1a48b894b76b40aa1fc8b7 [sys.catalog]: This master's current role is: LEADER
I20260809 09:35:06.627882 6831 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 09:35:06.631247 6831 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 09:35:06.631697 6810 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786267941944493-20982-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260809 09:35:06.641988 6831 catalog_manager.cc:1347] Generated new cluster ID: 32044a65eb5349a0a1ba4814148da68f
I20260809 09:35:06.642081 6831 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 09:35:06.659675 6831 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 09:35:06.660677 6831 catalog_manager.cc:1504] Loading token signing keys...
I20260809 09:35:06.667680 6831 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5c09265aec1a48b894b76b40aa1fc8b7: Generated new TSK 0
I20260809 09:35:06.668412 6831 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.20.125.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.125.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.125.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.129.keytab.
Entry for principal kudu/127.20.125.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.129.keytab.
I20260809 09:35:07.377444 20982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
/tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786267941944493-20982-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786267941944493-20982-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786267941944493-20982-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786267941944493-20982-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.20.125.148:57127/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.125.129:0
--local_ip_for_outbound_sockets=127.20.125.129
--webserver_interface=127.20.125.129
--webserver_port=0
--tserver_master_addrs=127.20.125.190:42791
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.129.keytab
--principal=kudu/127.20.125.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.125.148:33717
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786267941944493-20982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786267941944493-20982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786267941944493-20982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 09:35:07.528973 6873 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 09:35:07.529233 6873 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 09:35:07.529276 6873 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 09:35:07.533464 6873 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 09:35:07.533526 6873 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 09:35:07.533586 6873 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.125.129
I20260809 09:35:07.539397 6873 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.125.148:33717
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786267941944493-20982-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786267941944493-20982-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.20.125.148:57127/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.129.keytab
--principal=kudu/127.20.125.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.125.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786267941944493-20982-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.125.129
--webserver_port=0
--tserver_master_addrs=127.20.125.190:42791
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6873
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.125.129
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786267941944493-20982-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 08:38:55 UTC on ccycloud.quasar-njjleu.root.comops.site
build id 4575935
I20260809 09:35:07.540542 6873 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 09:35:07.541642 6873 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 09:35:07.554450 6885 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:35:07.555022 6884 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:35:07.555074 6889 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:35:07.555768 6873 server_base.cc:1034] running on GCE node
I20260809 09:35:07.556461 6873 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 09:35:07.557495 6873 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 09:35:07.558749 6873 hybrid_clock.cc:648] HybridClock initialized: now 1786268107558677 us; error 89 us; skew 500 ppm
Aug 09 09:35:07 dist-test-slave-7kzw krb5kdc[6076](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268107, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 09:35:07.562837 6873 init.cc:377] Logged in from keytab as kudu/127.20.125.129@KRBTEST.COM (short username kudu)
I20260809 09:35:07.563971 6873 webserver.cc:460] Webserver started at http://127.20.125.129:38653/ using document root <none> and password file <none>
I20260809 09:35:07.564714 6873 fs_manager.cc:359] Metadata directory not provided
I20260809 09:35:07.564812 6873 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 09:35:07.565052 6873 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 09:35:07 dist-test-slave-7kzw krb5kdc[6076](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268107, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.129@KRBTEST.COM for HTTP/127.20.125.148@KRBTEST.COM
I20260809 09:35:07.581669 6873 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786267941944493-20982-0/minicluster-data/ts-0/data/instance:
uuid: "3003436f17b94848a8b90679923b7fa2"
format_stamp: "Formatted at 2026-08-09 09:35:07 on dist-test-slave-7kzw"
server_key: "db460a3deadffd70a6564427e45040f9"
server_key_iv: "1fb83315506432a6d93449f27b85e402"
server_key_version: "kuduclusterkey@0"
I20260809 09:35:07.582620 6873 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786267941944493-20982-0/minicluster-data/ts-0/wal/instance:
uuid: "3003436f17b94848a8b90679923b7fa2"
format_stamp: "Formatted at 2026-08-09 09:35:07 on dist-test-slave-7kzw"
server_key: "db460a3deadffd70a6564427e45040f9"
server_key_iv: "1fb83315506432a6d93449f27b85e402"
server_key_version: "kuduclusterkey@0"
I20260809 09:35:07.592082 6873 fs_manager.cc:691] Time spent creating directory manager: real 0.009s user 0.004s sys 0.000s
I20260809 09:35:07.610697 6902 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 09:35:07.612382 6873 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260809 09:35:07.612512 6873 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786267941944493-20982-0/minicluster-data/ts-0/data,/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786267941944493-20982-0/minicluster-data/ts-0/wal
uuid: "3003436f17b94848a8b90679923b7fa2"
format_stamp: "Formatted at 2026-08-09 09:35:07 on dist-test-slave-7kzw"
server_key: "db460a3deadffd70a6564427e45040f9"
server_key_iv: "1fb83315506432a6d93449f27b85e402"
server_key_version: "kuduclusterkey@0"
I20260809 09:35:07.612649 6873 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786267941944493-20982-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786267941944493-20982-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786267941944493-20982-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 09:35:07.641841 6873 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 09:35:07.645813 6873 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 09:35:07.646117 6873 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 09:35:07.646890 6873 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 09:35:07.648036 6873 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 09:35:07.648113 6873 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 09:35:07.648172 6873 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 09:35:07.648202 6873 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 09:35:07.657995 6873 rpc_server.cc:307] RPC server started. Bound to: 127.20.125.129:39843
I20260809 09:35:07.659288 6873 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786267941944493-20982-0/minicluster-data/ts-0/data/info.pb
I20260809 09:35:07.660161 20982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu as pid 6873
I20260809 09:35:07.660305 20982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786267941944493-20982-0/minicluster-data/ts-0/wal/instance
I20260809 09:35:07.660666 20982 external_mini_cluster.cc:1428] Setting key f16c2017c0f5d75a8c7c6e0dce7a6ad3
I20260809 09:35:07.674207 7072 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.125.129:39843 every 8 connection(s)
Aug 09 09:35:07 dist-test-slave-7kzw krb5kdc[6076](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268107, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.129@KRBTEST.COM for kudu/127.20.125.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.20.125.130@KRBTEST.COM; defaulting to no policy
I20260809 09:35:07.687057 7080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:35:07.667773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.129:44011 (local address 127.20.125.190:42791)
0809 09:35:07.668339 (+ 566us) server_negotiation.cc:206] Beginning negotiation
0809 09:35:07.668347 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 09:35:07.670103 (+ 1756us) server_negotiation.cc:407] Connection header received
0809 09:35:07.670155 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:35:07.670158 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:35:07.670218 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:35:07.670304 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:35:07.673757 (+ 3453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:35:07.674704 (+ 947us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:35:07.677947 (+ 3243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:35:07.678328 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:35:07.678416 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:35:07.682180 (+ 3764us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:35:07.682187 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:35:07.682191 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:35:07.682223 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:35:07.684349 (+ 2126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:35:07.685050 (+ 701us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:35:07.685053 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:35:07.685056 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:35:07.685153 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:35:07.685596 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:35:07.685600 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:35:07.685602 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:35:07.685852 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:35:07.685946 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0809 09:35:07.686692 (+ 746us) server_negotiation.cc:299] Negotiation successful
0809 09:35:07.686841 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":486,"thread_start_us":105,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.125.130@KRBTEST.COM" created.
I20260809 09:35:07.688364 7079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:35:07.666669 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:42791 (local address 127.20.125.129:44011)
0809 09:35:07.668625 (+ 1956us) negotiation.cc:107] Waiting for socket to connect
0809 09:35:07.668687 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0809 09:35:07.669766 (+ 1079us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:35:07.670731 (+ 965us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:35:07.670741 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:35:07.671665 (+ 924us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:35:07.672384 (+ 719us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:35:07.672405 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:35:07.674882 (+ 2477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:35:07.674886 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:35:07.676335 (+ 1449us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:35:07.676349 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:35:07.678718 (+ 2369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:35:07.678722 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:35:07.678937 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:35:07.679964 (+ 1027us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 09:35:07.679991 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:35:07.681935 (+ 1944us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 09:35:07.684501 (+ 2566us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:35:07.684508 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:35:07.684526 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:35:07.684920 (+ 394us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:35:07.685256 (+ 336us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:35:07.685259 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:35:07.685261 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:35:07.685444 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:35:07.686020 (+ 576us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 09:35:07.686027 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 09:35:07.686362 (+ 335us) client_negotiation.cc:769] Sending connection context
0809 09:35:07.686608 (+ 246us) client_negotiation.cc:240] Negotiation successful
0809 09:35:07.686961 (+ 353us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1725,"thread_start_us":160,"threads_started":1}
I20260809 09:35:07.690106 7073 heartbeater.cc:347] Connected to a master server at 127.20.125.190:42791
I20260809 09:35:07.690371 7073 heartbeater.cc:464] Registering TS with master...
I20260809 09:35:07.690933 7073 heartbeater.cc:511] Master 127.20.125.190:42791 requested a full tablet report, sending...
I20260809 09:35:07.692631 6756 ts_manager.cc:194] Registered new tserver with Master: 3003436f17b94848a8b90679923b7fa2 (127.20.125.129:39843)
I20260809 09:35:07.694259 6756 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.125.129@KRBTEST.COM'} at 127.20.125.129:44011
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.125.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.130.keytab.
Entry for principal kudu/127.20.125.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.130.keytab.
I20260809 09:35:07.710366 20982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
/tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786267941944493-20982-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786267941944493-20982-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786267941944493-20982-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786267941944493-20982-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.20.125.148:57127/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.125.130:0
--local_ip_for_outbound_sockets=127.20.125.130
--webserver_interface=127.20.125.130
--webserver_port=0
--tserver_master_addrs=127.20.125.190:42791
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.130.keytab
--principal=kudu/127.20.125.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.125.148:33717
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786267941944493-20982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786267941944493-20982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786267941944493-20982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 09:35:07 dist-test-slave-7kzw krb5kdc[6076](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786268107, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260809 09:35:07.843559 7086 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 09:35:07.843878 7086 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 09:35:07.844013 7086 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 09:35:07.849434 7086 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 09:35:07.849509 7086 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 09:35:07.849570 7086 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.125.130
I20260809 09:35:07.855738 7086 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.125.148:33717
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786267941944493-20982-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786267941944493-20982-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.20.125.148:57127/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.130.keytab
--principal=kudu/127.20.125.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.125.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786267941944493-20982-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.125.130
--webserver_port=0
--tserver_master_addrs=127.20.125.190:42791
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7086
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.125.130
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786267941944493-20982-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 08:38:55 UTC on ccycloud.quasar-njjleu.root.comops.site
build id 4575935
I20260809 09:35:07.856966 7086 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 09:35:07.858026 7086 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 09:35:07.868705 7094 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:35:07.868705 7096 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:35:07.868963 7098 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:35:07.869894 7086 server_base.cc:1034] running on GCE node
I20260809 09:35:07.870360 7086 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 09:35:07.871150 7086 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 09:35:07.872362 7086 hybrid_clock.cc:648] HybridClock initialized: now 1786268107872327 us; error 48 us; skew 500 ppm
Aug 09 09:35:07 dist-test-slave-7kzw krb5kdc[6076](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268107, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 09:35:07.875641 7086 init.cc:377] Logged in from keytab as kudu/127.20.125.130@KRBTEST.COM (short username kudu)
I20260809 09:35:07.876696 7086 webserver.cc:460] Webserver started at http://127.20.125.130:46441/ using document root <none> and password file <none>
I20260809 09:35:07.877373 7086 fs_manager.cc:359] Metadata directory not provided
I20260809 09:35:07.877480 7086 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 09:35:07.877741 7086 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 09:35:07 dist-test-slave-7kzw krb5kdc[6076](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268107, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.130@KRBTEST.COM for HTTP/127.20.125.148@KRBTEST.COM
I20260809 09:35:07.894166 7086 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786267941944493-20982-0/minicluster-data/ts-1/data/instance:
uuid: "eaa1c788ae2249848553125ad80445ca"
format_stamp: "Formatted at 2026-08-09 09:35:07 on dist-test-slave-7kzw"
server_key: "db6b92066f43aaa77b257dd25c77768c"
server_key_iv: "f1f80902484795248d4ed5a68e659fd5"
server_key_version: "kuduclusterkey@0"
I20260809 09:35:07.895036 7086 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786267941944493-20982-0/minicluster-data/ts-1/wal/instance:
uuid: "eaa1c788ae2249848553125ad80445ca"
format_stamp: "Formatted at 2026-08-09 09:35:07 on dist-test-slave-7kzw"
server_key: "db6b92066f43aaa77b257dd25c77768c"
server_key_iv: "f1f80902484795248d4ed5a68e659fd5"
server_key_version: "kuduclusterkey@0"
I20260809 09:35:07.900823 7086 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260809 09:35:07.923205 7106 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 09:35:07.925122 7086 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260809 09:35:07.925276 7086 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786267941944493-20982-0/minicluster-data/ts-1/data,/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786267941944493-20982-0/minicluster-data/ts-1/wal
uuid: "eaa1c788ae2249848553125ad80445ca"
format_stamp: "Formatted at 2026-08-09 09:35:07 on dist-test-slave-7kzw"
server_key: "db6b92066f43aaa77b257dd25c77768c"
server_key_iv: "f1f80902484795248d4ed5a68e659fd5"
server_key_version: "kuduclusterkey@0"
I20260809 09:35:07.925450 7086 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786267941944493-20982-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786267941944493-20982-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786267941944493-20982-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 09:35:07.942919 7086 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 09:35:07.946820 7086 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 09:35:07.947033 7086 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 09:35:07.947829 7086 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 09:35:07.949018 7086 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 09:35:07.949092 7086 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 09:35:07.949156 7086 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 09:35:07.949198 7086 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 09:35:07.961390 7086 rpc_server.cc:307] RPC server started. Bound to: 127.20.125.130:35377
I20260809 09:35:07.962746 7086 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786267941944493-20982-0/minicluster-data/ts-1/data/info.pb
I20260809 09:35:07.963928 7275 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.125.130:35377 every 8 connection(s)
I20260809 09:35:07.964378 20982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu as pid 7086
I20260809 09:35:07.964582 20982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786267941944493-20982-0/minicluster-data/ts-1/wal/instance
I20260809 09:35:07.964921 20982 external_mini_cluster.cc:1428] Setting key f141b82c4569808d510f57f8765d5ca6
WARNING: no policy specified for kudu/127.20.125.131@KRBTEST.COM; defaulting to no policy
Aug 09 09:35:07 dist-test-slave-7kzw krb5kdc[6076](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268107, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.130@KRBTEST.COM for kudu/127.20.125.190@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.125.131@KRBTEST.COM" created.
I20260809 09:35:07.989346 7080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:35:07.973286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.130:55797 (local address 127.20.125.190:42791)
0809 09:35:07.973427 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0809 09:35:07.973432 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 09:35:07.975082 (+ 1650us) server_negotiation.cc:407] Connection header received
0809 09:35:07.976490 (+ 1408us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:35:07.976494 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:35:07.976553 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:35:07.976621 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:35:07.978386 (+ 1765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:35:07.979272 (+ 886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:35:07.981035 (+ 1763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:35:07.981390 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:35:07.981482 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:35:07.984755 (+ 3273us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:35:07.984759 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:35:07.984762 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:35:07.984791 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:35:07.986697 (+ 1906us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:35:07.987388 (+ 691us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:35:07.987392 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:35:07.987394 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:35:07.987491 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:35:07.987954 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:35:07.987958 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:35:07.987960 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:35:07.988203 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:35:07.988282 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0809 09:35:07.989014 (+ 732us) server_negotiation.cc:299] Negotiation successful
0809 09:35:07.989158 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260809 09:35:07.991076 7279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:35:07.972033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:42791 (local address 127.20.125.130:55797)
0809 09:35:07.974901 (+ 2868us) negotiation.cc:107] Waiting for socket to connect
0809 09:35:07.974964 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0809 09:35:07.976213 (+ 1249us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:35:07.977070 (+ 857us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:35:07.977080 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:35:07.977453 (+ 373us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:35:07.978186 (+ 733us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:35:07.978208 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:35:07.979455 (+ 1247us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:35:07.979458 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:35:07.980880 (+ 1422us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:35:07.980892 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:35:07.981723 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:35:07.981727 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:35:07.981935 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:35:07.982666 (+ 731us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 09:35:07.982690 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:35:07.984521 (+ 1831us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 09:35:07.986840 (+ 2319us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:35:07.986846 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:35:07.986864 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:35:07.987239 (+ 375us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:35:07.987637 (+ 398us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:35:07.987640 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:35:07.987642 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:35:07.987828 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:35:07.988366 (+ 538us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 09:35:07.988373 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 09:35:07.988712 (+ 339us) client_negotiation.cc:769] Sending connection context
0809 09:35:07.989541 (+ 829us) client_negotiation.cc:240] Negotiation successful
0809 09:35:07.989921 (+ 380us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2624,"thread_start_us":167,"threads_started":1}
I20260809 09:35:07.992798 7276 heartbeater.cc:347] Connected to a master server at 127.20.125.190:42791
I20260809 09:35:07.993078 7276 heartbeater.cc:464] Registering TS with master...
I20260809 09:35:07.993721 7276 heartbeater.cc:511] Master 127.20.125.190:42791 requested a full tablet report, sending...
I20260809 09:35:07.995011 6756 ts_manager.cc:194] Registered new tserver with Master: eaa1c788ae2249848553125ad80445ca (127.20.125.130:35377)
I20260809 09:35:07.995785 6756 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.125.130@KRBTEST.COM'} at 127.20.125.130:55797
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.125.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.131.keytab.
Entry for principal kudu/127.20.125.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.131.keytab.
I20260809 09:35:08.010774 20982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
/tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786267941944493-20982-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786267941944493-20982-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786267941944493-20982-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786267941944493-20982-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.20.125.148:57127/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.125.131:0
--local_ip_for_outbound_sockets=127.20.125.131
--webserver_interface=127.20.125.131
--webserver_port=0
--tserver_master_addrs=127.20.125.190:42791
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.131.keytab
--principal=kudu/127.20.125.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.125.148:33717
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786267941944493-20982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786267941944493-20982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786267941944493-20982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 09:35:08.170420 7281 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 09:35:08.170790 7281 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 09:35:08.170925 7281 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 09:35:08.175237 7281 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 09:35:08.175331 7281 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 09:35:08.175437 7281 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.125.131
I20260809 09:35:08.182348 7281 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.125.148:33717
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786267941944493-20982-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786267941944493-20982-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.20.125.148:57127/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.131.keytab
--principal=kudu/127.20.125.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.125.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786267941944493-20982-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.125.131
--webserver_port=0
--tserver_master_addrs=127.20.125.190:42791
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7281
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.125.131
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786267941944493-20982-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 08:38:55 UTC on ccycloud.quasar-njjleu.root.comops.site
build id 4575935
I20260809 09:35:08.183578 7281 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 09:35:08.185017 7281 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 09:35:08.195883 7291 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:35:08.196019 7293 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:35:08.195883 7289 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:35:08.197450 7281 server_base.cc:1034] running on GCE node
I20260809 09:35:08.198031 7281 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 09:35:08.200743 7281 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 09:35:08.202136 7281 hybrid_clock.cc:648] HybridClock initialized: now 1786268108202059 us; error 111 us; skew 500 ppm
Aug 09 09:35:08 dist-test-slave-7kzw krb5kdc[6076](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268108, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 09:35:08.205600 7281 init.cc:377] Logged in from keytab as kudu/127.20.125.131@KRBTEST.COM (short username kudu)
I20260809 09:35:08.206825 7281 webserver.cc:460] Webserver started at http://127.20.125.131:46329/ using document root <none> and password file <none>
I20260809 09:35:08.207641 7281 fs_manager.cc:359] Metadata directory not provided
I20260809 09:35:08.207767 7281 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 09:35:08.208067 7281 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 09:35:08 dist-test-slave-7kzw krb5kdc[6076](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268108, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.131@KRBTEST.COM for HTTP/127.20.125.148@KRBTEST.COM
I20260809 09:35:08.224675 7281 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786267941944493-20982-0/minicluster-data/ts-2/data/instance:
uuid: "9da8b5f247aa45c4a622ee3ba870c322"
format_stamp: "Formatted at 2026-08-09 09:35:08 on dist-test-slave-7kzw"
server_key: "1bcfd18646a46c7ff1f45c80a5d8a21f"
server_key_iv: "0095238687bcf8cb0648a4939a18ed64"
server_key_version: "kuduclusterkey@0"
I20260809 09:35:08.225585 7281 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786267941944493-20982-0/minicluster-data/ts-2/wal/instance:
uuid: "9da8b5f247aa45c4a622ee3ba870c322"
format_stamp: "Formatted at 2026-08-09 09:35:08 on dist-test-slave-7kzw"
server_key: "1bcfd18646a46c7ff1f45c80a5d8a21f"
server_key_iv: "0095238687bcf8cb0648a4939a18ed64"
server_key_version: "kuduclusterkey@0"
I20260809 09:35:08.232620 7281 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.008s sys 0.000s
I20260809 09:35:08.261236 7304 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 09:35:08.263108 7281 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260809 09:35:08.263265 7281 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786267941944493-20982-0/minicluster-data/ts-2/data,/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786267941944493-20982-0/minicluster-data/ts-2/wal
uuid: "9da8b5f247aa45c4a622ee3ba870c322"
format_stamp: "Formatted at 2026-08-09 09:35:08 on dist-test-slave-7kzw"
server_key: "1bcfd18646a46c7ff1f45c80a5d8a21f"
server_key_iv: "0095238687bcf8cb0648a4939a18ed64"
server_key_version: "kuduclusterkey@0"
I20260809 09:35:08.263427 7281 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786267941944493-20982-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786267941944493-20982-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786267941944493-20982-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 09:35:08.291574 7281 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 09:35:08.295531 7281 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 09:35:08.295748 7281 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 09:35:08.296504 7281 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 09:35:08.297605 7281 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 09:35:08.297678 7281 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 09:35:08.297750 7281 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 09:35:08.297783 7281 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 09:35:08 dist-test-slave-7kzw krb5kdc[6076](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268107, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.190@KRBTEST.COM for HTTP/127.20.125.148@KRBTEST.COM
I20260809 09:35:08.311553 7281 rpc_server.cc:307] RPC server started. Bound to: 127.20.125.131:34399
I20260809 09:35:08.312909 7281 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786267941944493-20982-0/minicluster-data/ts-2/data/info.pb
I20260809 09:35:08.313868 7466 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.125.131:34399 every 8 connection(s)
I20260809 09:35:08.319096 20982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu as pid 7281
I20260809 09:35:08.319221 20982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786267941944493-20982-0/minicluster-data/ts-2/wal/instance
I20260809 09:35:08.319533 20982 external_mini_cluster.cc:1428] Setting key 31e5fbac6c8e4655dbde76aa8ff28835
Aug 09 09:35:08 dist-test-slave-7kzw krb5kdc[6076](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268108, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.131@KRBTEST.COM for kudu/127.20.125.190@KRBTEST.COM
I20260809 09:35:08.333648 7080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:35:08.318306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.131:56241 (local address 127.20.125.190:42791)
0809 09:35:08.318433 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0809 09:35:08.318440 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 09:35:08.318523 (+ 83us) server_negotiation.cc:407] Connection header received
0809 09:35:08.319401 (+ 878us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:35:08.319408 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:35:08.319472 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:35:08.319529 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:35:08.321639 (+ 2110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:35:08.322642 (+ 1003us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:35:08.324519 (+ 1877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:35:08.324919 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:35:08.324999 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:35:08.328603 (+ 3604us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:35:08.328608 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:35:08.328611 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:35:08.328639 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:35:08.330986 (+ 2347us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:35:08.331642 (+ 656us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:35:08.331646 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:35:08.331648 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:35:08.331754 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:35:08.332206 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:35:08.332210 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:35:08.332212 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:35:08.332453 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:35:08.332529 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 09:35:08.333319 (+ 790us) server_negotiation.cc:299] Negotiation successful
0809 09:35:08.333464 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260809 09:35:08.335835 7473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:35:08.317054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:42791 (local address 127.20.125.131:56241)
0809 09:35:08.317753 (+ 699us) negotiation.cc:107] Waiting for socket to connect
0809 09:35:08.317810 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0809 09:35:08.319133 (+ 1323us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:35:08.319866 (+ 733us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:35:08.319872 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:35:08.320501 (+ 629us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:35:08.321395 (+ 894us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:35:08.321420 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:35:08.322856 (+ 1436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:35:08.322860 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:35:08.324363 (+ 1503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:35:08.324375 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:35:08.325214 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:35:08.325218 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:35:08.325441 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:35:08.326203 (+ 762us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 09:35:08.326228 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:35:08.328400 (+ 2172us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 09:35:08.331113 (+ 2713us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:35:08.331120 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:35:08.331137 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:35:08.331523 (+ 386us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:35:08.331876 (+ 353us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:35:08.331880 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:35:08.331882 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:35:08.332080 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:35:08.332619 (+ 539us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 09:35:08.332626 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 09:35:08.333002 (+ 376us) client_negotiation.cc:769] Sending connection context
0809 09:35:08.334151 (+ 1149us) client_negotiation.cc:240] Negotiation successful
0809 09:35:08.334586 (+ 435us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":430,"thread_start_us":202,"threads_started":1}
I20260809 09:35:08.337796 7467 heartbeater.cc:347] Connected to a master server at 127.20.125.190:42791
I20260809 09:35:08.338071 7467 heartbeater.cc:464] Registering TS with master...
I20260809 09:35:08.338723 7467 heartbeater.cc:511] Master 127.20.125.190:42791 requested a full tablet report, sending...
I20260809 09:35:08.339900 6756 ts_manager.cc:194] Registered new tserver with Master: 9da8b5f247aa45c4a622ee3ba870c322 (127.20.125.131:34399)
I20260809 09:35:08.340665 6756 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.125.131@KRBTEST.COM'} at 127.20.125.131:56241
I20260809 09:35:08.347504 20982 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 09:35:08.348353 20982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu with pid 6873
I20260809 09:35:08.389580 20982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu with pid 7086
I20260809 09:35:08.401319 20982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu with pid 7281
I20260809 09:35:08.416059 20982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu with pid 6706
I20260809 09:35:08.433615 20982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
/tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786267941944493-20982-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786267941944493-20982-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786267941944493-20982-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786267941944493-20982-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.125.190:42791
--webserver_interface=127.20.125.190
--webserver_port=36387
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.190.keytab
--principal=kudu/127.20.125.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.125.148:33717
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.125.190:42791
--ranger_config_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786267941944493-20982-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.20.125.148:57127/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.20.125.148:57127/kms with env {KRB5CCNAME=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786267941944493-20982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786267941944493-20982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786267941944493-20982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 09:35:08.552958 7474 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 09:35:08.553220 7474 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 09:35:08.553260 7474 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 09:35:08.557374 7474 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 09:35:08.557436 7474 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 09:35:08.557456 7474 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 09:35:08.557471 7474 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 09:35:08.557525 7474 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 09:35:08.563095 7474 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.125.148:33717
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786267941944493-20982-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786267941944493-20982-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.20.125.148:57127/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.20.125.190:42791
--ranger_config_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786267941944493-20982-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.190.keytab
--principal=kudu/127.20.125.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.125.190:42791
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786267941944493-20982-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.125.190
--webserver_port=36387
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7474
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786267941944493-20982-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 08:38:55 UTC on ccycloud.quasar-njjleu.root.comops.site
build id 4575935
I20260809 09:35:08.564172 7474 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 09:35:08.565279 7474 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 09:35:08.574431 7474 server_base.cc:1034] running on GCE node
W20260809 09:35:08.574369 7482 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:35:08.574369 7484 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:35:08.574393 7481 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:35:08.575423 7474 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 09:35:08.576581 7474 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 09:35:08.577765 7474 hybrid_clock.cc:648] HybridClock initialized: now 1786268108577745 us; error 33 us; skew 500 ppm
Aug 09 09:35:08 dist-test-slave-7kzw krb5kdc[6076](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268108, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 09:35:08.580794 7474 init.cc:377] Logged in from keytab as kudu/127.20.125.190@KRBTEST.COM (short username kudu)
I20260809 09:35:08.581758 7474 webserver.cc:460] Webserver started at http://127.20.125.190:36387/ using document root <none> and password file <none>
I20260809 09:35:08.582325 7474 fs_manager.cc:359] Metadata directory not provided
I20260809 09:35:08.582430 7474 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 09 09:35:08 dist-test-slave-7kzw krb5kdc[6076](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268108, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.190@KRBTEST.COM for HTTP/127.20.125.148@KRBTEST.COM
I20260809 09:35:08.630739 7474 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260809 09:35:08.633512 7499 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 09:35:08.635236 7474 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 09:35:08.635421 7474 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786267941944493-20982-0/minicluster-data/master-0/data,/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786267941944493-20982-0/minicluster-data/master-0/wal
uuid: "5c09265aec1a48b894b76b40aa1fc8b7"
format_stamp: "Formatted at 2026-08-09 09:35:06 on dist-test-slave-7kzw"
server_key: "7aade926d57b40ca3f567b26b42433e3"
server_key_iv: "469bd569b441fbe6a632f5c33fb617c6"
server_key_version: "kuduclusterkey@0"
I20260809 09:35:08.636013 7474 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786267941944493-20982-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786267941944493-20982-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786267941944493-20982-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 09:35:08.649863 7474 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 09:35:08.653237 7474 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 09:35:08.653460 7474 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 09:35:08.661635 7474 rpc_server.cc:307] RPC server started. Bound to: 127.20.125.190:42791
I20260809 09:35:08.661733 7573 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.125.190:42791 every 8 connection(s)
I20260809 09:35:08.662945 7474 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786267941944493-20982-0/minicluster-data/master-0/data/info.pb
I20260809 09:35:08.667487 7574 sys_catalog.cc:263] Verifying existing consensus state
I20260809 09:35:08.669263 7574 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5c09265aec1a48b894b76b40aa1fc8b7: Bootstrap starting.
I20260809 09:35:08.672072 20982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu as pid 7474
I20260809 09:35:08.672642 20982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
/tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786267941944493-20982-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786267941944493-20982-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786267941944493-20982-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786267941944493-20982-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.20.125.148:57127/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.125.129:39843
--local_ip_for_outbound_sockets=127.20.125.129
--tserver_master_addrs=127.20.125.190:42791
--webserver_port=38653
--webserver_interface=127.20.125.129
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.129.keytab
--principal=kudu/127.20.125.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.125.148:33717
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.20.125.148:57127/kms with env {KRB5CCNAME=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786267941944493-20982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786267941944493-20982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786267941944493-20982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 09:35:08.686952 7574 log.cc:824] T 00000000000000000000000000000000 P 5c09265aec1a48b894b76b40aa1fc8b7: Log is configured to *not* fsync() on all Append() calls
I20260809 09:35:08.694171 7574 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5c09265aec1a48b894b76b40aa1fc8b7: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 09:35:08.694669 7574 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5c09265aec1a48b894b76b40aa1fc8b7: Bootstrap complete.
I20260809 09:35:08.698890 7574 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5c09265aec1a48b894b76b40aa1fc8b7 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c09265aec1a48b894b76b40aa1fc8b7" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 42791 } }
I20260809 09:35:08.699293 7574 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5c09265aec1a48b894b76b40aa1fc8b7 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c09265aec1a48b894b76b40aa1fc8b7, State: Initialized, Role: FOLLOWER
I20260809 09:35:08.699862 7574 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5c09265aec1a48b894b76b40aa1fc8b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c09265aec1a48b894b76b40aa1fc8b7" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 42791 } }
I20260809 09:35:08.699982 7574 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5c09265aec1a48b894b76b40aa1fc8b7 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 09:35:08.700075 7574 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5c09265aec1a48b894b76b40aa1fc8b7 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 09:35:08.700174 7574 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5c09265aec1a48b894b76b40aa1fc8b7 [term 1 FOLLOWER]: Advancing to term 2
I20260809 09:35:08.702442 7574 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5c09265aec1a48b894b76b40aa1fc8b7 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c09265aec1a48b894b76b40aa1fc8b7" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 42791 } }
I20260809 09:35:08.702852 7574 leader_election.cc:302] T 00000000000000000000000000000000 P 5c09265aec1a48b894b76b40aa1fc8b7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5c09265aec1a48b894b76b40aa1fc8b7; no voters:
I20260809 09:35:08.703224 7574 leader_election.cc:288] T 00000000000000000000000000000000 P 5c09265aec1a48b894b76b40aa1fc8b7 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260809 09:35:08.703327 7581 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5c09265aec1a48b894b76b40aa1fc8b7 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 09:35:08.703508 7581 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5c09265aec1a48b894b76b40aa1fc8b7 [term 2 LEADER]: Becoming Leader. State: Replica: 5c09265aec1a48b894b76b40aa1fc8b7, State: Running, Role: LEADER
I20260809 09:35:08.703806 7581 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5c09265aec1a48b894b76b40aa1fc8b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c09265aec1a48b894b76b40aa1fc8b7" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 42791 } }
I20260809 09:35:08.704512 7574 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5c09265aec1a48b894b76b40aa1fc8b7 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 09:35:08.705219 7582 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5c09265aec1a48b894b76b40aa1fc8b7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "5c09265aec1a48b894b76b40aa1fc8b7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c09265aec1a48b894b76b40aa1fc8b7" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 42791 } } }
I20260809 09:35:08.705358 7582 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5c09265aec1a48b894b76b40aa1fc8b7 [sys.catalog]: This master's current role is: LEADER
I20260809 09:35:08.705679 7583 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5c09265aec1a48b894b76b40aa1fc8b7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5c09265aec1a48b894b76b40aa1fc8b7. Latest consensus state: current_term: 2 leader_uuid: "5c09265aec1a48b894b76b40aa1fc8b7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c09265aec1a48b894b76b40aa1fc8b7" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 42791 } } }
I20260809 09:35:08.705767 7583 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5c09265aec1a48b894b76b40aa1fc8b7 [sys.catalog]: This master's current role is: LEADER
I20260809 09:35:08.706220 7590 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 09:35:08.709584 7590 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 09:35:08.710222 7574 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786267941944493-20982-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260809 09:35:08.710251 7590 catalog_manager.cc:1259] Loaded cluster ID: 32044a65eb5349a0a1ba4814148da68f
I20260809 09:35:08.710449 7590 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 09:35:08.712883 7590 catalog_manager.cc:1504] Loading token signing keys...
I20260809 09:35:08.713568 7590 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 5c09265aec1a48b894b76b40aa1fc8b7: Loaded TSK: 0
I20260809 09:35:08.714814 7590 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260809 09:35:08.821597 7578 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 09:35:08.821962 7578 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 09:35:08.822027 7578 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 09:35:08.828528 7578 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 09:35:08.828617 7578 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 09:35:08.828711 7578 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.125.129
I20260809 09:35:08.836522 7578 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.125.148:33717
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786267941944493-20982-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786267941944493-20982-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.20.125.148:57127/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.129.keytab
--principal=kudu/127.20.125.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.125.129:39843
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786267941944493-20982-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.125.129
--webserver_port=38653
--tserver_master_addrs=127.20.125.190:42791
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7578
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.125.129
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786267941944493-20982-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 08:38:55 UTC on ccycloud.quasar-njjleu.root.comops.site
build id 4575935
I20260809 09:35:08.837991 7578 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 09:35:08.839484 7578 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 09:35:08.852619 7633 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:35:08.852883 7630 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:35:08.852630 7631 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:35:08.853159 7578 server_base.cc:1034] running on GCE node
I20260809 09:35:08.853832 7578 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 09:35:08.854712 7578 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 09:35:08.855924 7578 hybrid_clock.cc:648] HybridClock initialized: now 1786268108855897 us; error 44 us; skew 500 ppm
Aug 09 09:35:08 dist-test-slave-7kzw krb5kdc[6076](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268108, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 09:35:08.859723 7578 init.cc:377] Logged in from keytab as kudu/127.20.125.129@KRBTEST.COM (short username kudu)
I20260809 09:35:08.861116 7578 webserver.cc:460] Webserver started at http://127.20.125.129:38653/ using document root <none> and password file <none>
I20260809 09:35:08.861882 7578 fs_manager.cc:359] Metadata directory not provided
I20260809 09:35:08.862006 7578 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 09 09:35:08 dist-test-slave-7kzw krb5kdc[6076](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268108, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.129@KRBTEST.COM for HTTP/127.20.125.148@KRBTEST.COM
I20260809 09:35:08.887377 7578 fs_manager.cc:709] Time spent opening directory manager: real 0.007s user 0.004s sys 0.000s
I20260809 09:35:08.891958 7642 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 09:35:08.893770 7578 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.003s sys 0.000s
I20260809 09:35:08.893920 7578 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786267941944493-20982-0/minicluster-data/ts-0/data,/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786267941944493-20982-0/minicluster-data/ts-0/wal
uuid: "3003436f17b94848a8b90679923b7fa2"
format_stamp: "Formatted at 2026-08-09 09:35:07 on dist-test-slave-7kzw"
server_key: "db460a3deadffd70a6564427e45040f9"
server_key_iv: "1fb83315506432a6d93449f27b85e402"
server_key_version: "kuduclusterkey@0"
I20260809 09:35:08.894761 7578 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786267941944493-20982-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786267941944493-20982-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786267941944493-20982-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 09:35:08.911837 7578 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 09:35:08.915177 7578 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 09:35:08.915405 7578 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 09:35:08.916180 7578 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 09:35:08.917193 7578 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 09:35:08.917270 7578 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 09:35:08.917351 7578 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 09:35:08.917398 7578 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 09:35:08.929386 7578 rpc_server.cc:307] RPC server started. Bound to: 127.20.125.129:39843
I20260809 09:35:08.929977 7794 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.125.129:39843 every 8 connection(s)
I20260809 09:35:08.931010 7578 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786267941944493-20982-0/minicluster-data/ts-0/data/info.pb
I20260809 09:35:08.936829 20982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu as pid 7578
I20260809 09:35:08.937430 20982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
/tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786267941944493-20982-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786267941944493-20982-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786267941944493-20982-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786267941944493-20982-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.20.125.148:57127/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.125.130:35377
--local_ip_for_outbound_sockets=127.20.125.130
--tserver_master_addrs=127.20.125.190:42791
--webserver_port=46441
--webserver_interface=127.20.125.130
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.130.keytab
--principal=kudu/127.20.125.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.125.148:33717
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.20.125.148:57127/kms with env {KRB5CCNAME=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786267941944493-20982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786267941944493-20982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786267941944493-20982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 09:35:08 dist-test-slave-7kzw krb5kdc[6076](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268108, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.129@KRBTEST.COM for kudu/127.20.125.190@KRBTEST.COM
I20260809 09:35:08.969511 7798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:35:08.935005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:42791 (local address 127.20.125.129:54151)
0809 09:35:08.936581 (+ 1576us) negotiation.cc:107] Waiting for socket to connect
0809 09:35:08.936653 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0809 09:35:08.939752 (+ 3099us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:35:08.942305 (+ 2553us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:35:08.942316 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:35:08.942779 (+ 463us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:35:08.943630 (+ 851us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:35:08.943654 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:35:08.946007 (+ 2353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:35:08.946010 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:35:08.947649 (+ 1639us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:35:08.947663 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:35:08.949019 (+ 1356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:35:08.949023 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:35:08.949223 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:35:08.950082 (+ 859us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 09:35:08.950105 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:35:08.952219 (+ 2114us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 09:35:08.955048 (+ 2829us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:35:08.955055 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:35:08.955071 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:35:08.955441 (+ 370us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:35:08.955926 (+ 485us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:35:08.955930 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:35:08.955932 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:35:08.956176 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:35:08.957102 (+ 926us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 09:35:08.957109 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 09:35:08.957430 (+ 321us) client_negotiation.cc:769] Sending connection context
0809 09:35:08.957655 (+ 225us) client_negotiation.cc:240] Negotiation successful
0809 09:35:08.968213 (+ 10558us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1319,"thread_start_us":143,"threads_started":1}
I20260809 09:35:08.971561 7797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:35:08.934848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.129:54151 (local address 127.20.125.190:42791)
0809 09:35:08.935606 (+ 758us) server_negotiation.cc:206] Beginning negotiation
0809 09:35:08.935632 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0809 09:35:08.936836 (+ 1204us) server_negotiation.cc:407] Connection header received
0809 09:35:08.940646 (+ 3810us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:35:08.940703 (+ 57us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:35:08.941297 (+ 594us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:35:08.941761 (+ 464us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:35:08.943943 (+ 2182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:35:08.945827 (+ 1884us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:35:08.947815 (+ 1988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:35:08.948576 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:35:08.948767 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:35:08.952466 (+ 3699us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:35:08.952474 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:35:08.952492 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:35:08.952527 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:35:08.954900 (+ 2373us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:35:08.955622 (+ 722us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:35:08.955629 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:35:08.955635 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:35:08.955801 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:35:08.956301 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:35:08.956305 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:35:08.956307 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:35:08.956889 (+ 582us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:35:08.957029 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0809 09:35:08.957968 (+ 939us) server_negotiation.cc:299] Negotiation successful
0809 09:35:08.970245 (+ 12277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":579,"thread_start_us":163,"threads_started":1}
I20260809 09:35:08.975530 7795 heartbeater.cc:347] Connected to a master server at 127.20.125.190:42791
I20260809 09:35:08.975832 7795 heartbeater.cc:464] Registering TS with master...
I20260809 09:35:08.976511 7795 heartbeater.cc:511] Master 127.20.125.190:42791 requested a full tablet report, sending...
W20260809 09:35:08.979060 7795 heartbeater.cc:644] Failed to heartbeat to 127.20.125.190:42791 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260809 09:35:08.979211 7795 heartbeater.cc:464] Registering TS with master...
I20260809 09:35:08.979454 7795 heartbeater.cc:511] Master 127.20.125.190:42791 requested a full tablet report, sending...
I20260809 09:35:08.980450 7795 heartbeater.cc:464] Registering TS with master...
I20260809 09:35:08.980645 7795 heartbeater.cc:511] Master 127.20.125.190:42791 requested a full tablet report, sending...
W20260809 09:35:08.981539 7795 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 09:35:09.119546 7799 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 09:35:09.119971 7799 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 09:35:09.120046 7799 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 09:35:09.126320 7799 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 09:35:09.126406 7799 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 09:35:09.126523 7799 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.125.130
I20260809 09:35:09.134769 7799 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.125.148:33717
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786267941944493-20982-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786267941944493-20982-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.20.125.148:57127/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.130.keytab
--principal=kudu/127.20.125.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.125.130:35377
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786267941944493-20982-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.125.130
--webserver_port=46441
--tserver_master_addrs=127.20.125.190:42791
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7799
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.125.130
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786267941944493-20982-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 08:38:55 UTC on ccycloud.quasar-njjleu.root.comops.site
build id 4575935
I20260809 09:35:09.136235 7799 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 09:35:09.137656 7799 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 09:35:09.150074 7817 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:35:09.150074 7809 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:35:09.150198 7808 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:35:09.150539 7799 server_base.cc:1034] running on GCE node
I20260809 09:35:09.151121 7799 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 09:35:09.151970 7799 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 09:35:09.153187 7799 hybrid_clock.cc:648] HybridClock initialized: now 1786268109153149 us; error 48 us; skew 500 ppm
Aug 09 09:35:09 dist-test-slave-7kzw krb5kdc[6076](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268109, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 09:35:09.157498 7799 init.cc:377] Logged in from keytab as kudu/127.20.125.130@KRBTEST.COM (short username kudu)
I20260809 09:35:09.158766 7799 webserver.cc:460] Webserver started at http://127.20.125.130:46441/ using document root <none> and password file <none>
I20260809 09:35:09.159504 7799 fs_manager.cc:359] Metadata directory not provided
I20260809 09:35:09.159632 7799 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 09 09:35:09 dist-test-slave-7kzw krb5kdc[6076](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268109, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.130@KRBTEST.COM for HTTP/127.20.125.148@KRBTEST.COM
I20260809 09:35:09.183266 7799 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.001s sys 0.002s
I20260809 09:35:09.187448 7829 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.001s user 0.000s sys 0.000s
I20260809 09:35:09.189136 7799 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260809 09:35:09.189373 7799 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786267941944493-20982-0/minicluster-data/ts-1/data,/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786267941944493-20982-0/minicluster-data/ts-1/wal
uuid: "eaa1c788ae2249848553125ad80445ca"
format_stamp: "Formatted at 2026-08-09 09:35:07 on dist-test-slave-7kzw"
server_key: "db6b92066f43aaa77b257dd25c77768c"
server_key_iv: "f1f80902484795248d4ed5a68e659fd5"
server_key_version: "kuduclusterkey@0"
I20260809 09:35:09.190132 7799 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786267941944493-20982-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786267941944493-20982-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786267941944493-20982-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 09:35:09.238862 7799 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 09:35:09.243168 7799 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 09:35:09.243403 7799 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 09:35:09.244292 7799 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 09:35:09.245519 7799 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 09:35:09.245610 7799 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 09:35:09.245697 7799 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 09:35:09.245730 7799 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 09:35:09.259608 7799 rpc_server.cc:307] RPC server started. Bound to: 127.20.125.130:35377
I20260809 09:35:09.259691 7995 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.125.130:35377 every 8 connection(s)
I20260809 09:35:09.261219 7799 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786267941944493-20982-0/minicluster-data/ts-1/data/info.pb
I20260809 09:35:09.269757 20982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu as pid 7799
I20260809 09:35:09.270340 20982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
/tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786267941944493-20982-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786267941944493-20982-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786267941944493-20982-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786267941944493-20982-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.20.125.148:57127/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.125.131:34399
--local_ip_for_outbound_sockets=127.20.125.131
--tserver_master_addrs=127.20.125.190:42791
--webserver_port=46329
--webserver_interface=127.20.125.131
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.131.keytab
--principal=kudu/127.20.125.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.125.148:33717
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.20.125.148:57127/kms with env {KRB5CCNAME=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786267941944493-20982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786267941944493-20982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786267941944493-20982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 09:35:09 dist-test-slave-7kzw krb5kdc[6076](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268109, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.130@KRBTEST.COM for kudu/127.20.125.190@KRBTEST.COM
I20260809 09:35:09.283854 7797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:35:09.263847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.130:39879 (local address 127.20.125.190:42791)
0809 09:35:09.265479 (+ 1632us) server_negotiation.cc:206] Beginning negotiation
0809 09:35:09.265485 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 09:35:09.265508 (+ 23us) server_negotiation.cc:407] Connection header received
0809 09:35:09.266561 (+ 1053us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:35:09.266565 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:35:09.266633 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:35:09.266743 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:35:09.269107 (+ 2364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:35:09.269984 (+ 877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:35:09.274067 (+ 4083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:35:09.274495 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:35:09.274597 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:35:09.278363 (+ 3766us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:35:09.278368 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:35:09.278371 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:35:09.278404 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:35:09.280641 (+ 2237us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:35:09.281688 (+ 1047us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:35:09.281692 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:35:09.281694 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:35:09.281799 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:35:09.282268 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:35:09.282273 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:35:09.282275 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:35:09.282562 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:35:09.282659 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0809 09:35:09.283463 (+ 804us) server_negotiation.cc:299] Negotiation successful
0809 09:35:09.283652 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1517}
I20260809 09:35:09.289361 7999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:35:09.264355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:42791 (local address 127.20.125.130:39879)
0809 09:35:09.265005 (+ 650us) negotiation.cc:107] Waiting for socket to connect
0809 09:35:09.265068 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0809 09:35:09.266229 (+ 1161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:35:09.267007 (+ 778us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:35:09.267017 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:35:09.267449 (+ 432us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:35:09.268869 (+ 1420us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:35:09.268894 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:35:09.272235 (+ 3341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:35:09.272243 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:35:09.273896 (+ 1653us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:35:09.273911 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:35:09.274911 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:35:09.274915 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:35:09.275148 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:35:09.275953 (+ 805us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 09:35:09.275986 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:35:09.278132 (+ 2146us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 09:35:09.280792 (+ 2660us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:35:09.280799 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:35:09.280818 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:35:09.281224 (+ 406us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:35:09.281935 (+ 711us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:35:09.281938 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:35:09.281941 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:35:09.282137 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:35:09.282752 (+ 615us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 09:35:09.282760 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 09:35:09.283115 (+ 355us) client_negotiation.cc:769] Sending connection context
0809 09:35:09.287597 (+ 4482us) client_negotiation.cc:240] Negotiation successful
0809 09:35:09.288101 (+ 504us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":388,"thread_start_us":188,"threads_started":1}
I20260809 09:35:09.291329 7996 heartbeater.cc:347] Connected to a master server at 127.20.125.190:42791
I20260809 09:35:09.291612 7996 heartbeater.cc:464] Registering TS with master...
I20260809 09:35:09.292276 7996 heartbeater.cc:511] Master 127.20.125.190:42791 requested a full tablet report, sending...
W20260809 09:35:09.294281 7996 heartbeater.cc:644] Failed to heartbeat to 127.20.125.190:42791 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260809 09:35:09.294428 7996 heartbeater.cc:464] Registering TS with master...
I20260809 09:35:09.294672 7996 heartbeater.cc:511] Master 127.20.125.190:42791 requested a full tablet report, sending...
I20260809 09:35:09.295644 7996 heartbeater.cc:464] Registering TS with master...
I20260809 09:35:09.295845 7996 heartbeater.cc:511] Master 127.20.125.190:42791 requested a full tablet report, sending...
W20260809 09:35:09.296686 7996 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 09:35:09.479179 8000 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 09:35:09.479550 8000 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 09:35:09.479672 8000 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 09:35:09.485600 8000 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 09:35:09.485687 8000 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 09:35:09.485769 8000 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.125.131
I20260809 09:35:09.493359 8000 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.125.148:33717
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786267941944493-20982-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786267941944493-20982-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.20.125.148:57127/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.131.keytab
--principal=kudu/127.20.125.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.125.131:34399
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786267941944493-20982-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.125.131
--webserver_port=46329
--tserver_master_addrs=127.20.125.190:42791
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8000
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.125.131
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786267941944493-20982-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 08:38:55 UTC on ccycloud.quasar-njjleu.root.comops.site
build id 4575935
I20260809 09:35:09.494808 8000 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 09:35:09.496272 8000 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 09:35:09.510001 8011 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:35:09.510486 8000 server_base.cc:1034] running on GCE node
W20260809 09:35:09.513175 8008 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:35:09.513572 8013 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:35:09.514199 8000 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 09:35:09.515134 8000 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 09:35:09.522521 8000 hybrid_clock.cc:648] HybridClock initialized: now 1786268109522424 us; error 117 us; skew 500 ppm
Aug 09 09:35:09 dist-test-slave-7kzw krb5kdc[6076](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268109, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 09:35:09.526598 8000 init.cc:377] Logged in from keytab as kudu/127.20.125.131@KRBTEST.COM (short username kudu)
I20260809 09:35:09.527899 8000 webserver.cc:460] Webserver started at http://127.20.125.131:46329/ using document root <none> and password file <none>
I20260809 09:35:09.528674 8000 fs_manager.cc:359] Metadata directory not provided
I20260809 09:35:09.528795 8000 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 09 09:35:09 dist-test-slave-7kzw krb5kdc[6076](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268109, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.131@KRBTEST.COM for HTTP/127.20.125.148@KRBTEST.COM
I20260809 09:35:09.553061 8000 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.003s sys 0.000s
I20260809 09:35:09.556100 8021 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 09:35:09.557817 8000 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 09:35:09.557955 8000 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786267941944493-20982-0/minicluster-data/ts-2/data,/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786267941944493-20982-0/minicluster-data/ts-2/wal
uuid: "9da8b5f247aa45c4a622ee3ba870c322"
format_stamp: "Formatted at 2026-08-09 09:35:08 on dist-test-slave-7kzw"
server_key: "1bcfd18646a46c7ff1f45c80a5d8a21f"
server_key_iv: "0095238687bcf8cb0648a4939a18ed64"
server_key_version: "kuduclusterkey@0"
I20260809 09:35:09.558693 8000 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786267941944493-20982-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786267941944493-20982-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786267941944493-20982-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 09:35:09.589781 8000 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 09:35:09.593815 8000 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 09:35:09.594043 8000 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 09:35:09.594882 8000 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 09:35:09.596179 8000 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 09:35:09.596259 8000 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 09:35:09.596375 8000 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 09:35:09.596414 8000 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 09:35:09.611234 8000 rpc_server.cc:307] RPC server started. Bound to: 127.20.125.131:34399
I20260809 09:35:09.612680 8000 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786267941944493-20982-0/minicluster-data/ts-2/data/info.pb
I20260809 09:35:09.613566 8183 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.125.131:34399 every 8 connection(s)
I20260809 09:35:09.620724 20982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu as pid 8000
Aug 09 09:35:09 dist-test-slave-7kzw krb5kdc[6076](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268109, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.131@KRBTEST.COM for kudu/127.20.125.190@KRBTEST.COM
I20260809 09:35:09.631735 7797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:35:09.616731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.131:58561 (local address 127.20.125.190:42791)
0809 09:35:09.616888 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0809 09:35:09.616893 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 09:35:09.616912 (+ 19us) server_negotiation.cc:407] Connection header received
0809 09:35:09.618205 (+ 1293us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:35:09.618222 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:35:09.618282 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:35:09.618340 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:35:09.620123 (+ 1783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:35:09.621017 (+ 894us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:35:09.622994 (+ 1977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:35:09.623371 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:35:09.623457 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:35:09.627128 (+ 3671us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:35:09.627134 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:35:09.627138 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:35:09.627171 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:35:09.629018 (+ 1847us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:35:09.629713 (+ 695us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:35:09.629717 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:35:09.629719 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:35:09.629814 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:35:09.630290 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:35:09.630294 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:35:09.630295 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:35:09.630584 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:35:09.630691 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0809 09:35:09.631406 (+ 715us) server_negotiation.cc:299] Negotiation successful
0809 09:35:09.631540 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260809 09:35:09.633541 8186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:35:09.615492 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:42791 (local address 127.20.125.131:58561)
0809 09:35:09.616122 (+ 630us) negotiation.cc:107] Waiting for socket to connect
0809 09:35:09.616192 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0809 09:35:09.617917 (+ 1725us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:35:09.618819 (+ 902us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:35:09.618829 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:35:09.619227 (+ 398us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:35:09.619930 (+ 703us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:35:09.619950 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:35:09.621297 (+ 1347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:35:09.621302 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:35:09.622836 (+ 1534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:35:09.622849 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:35:09.623683 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:35:09.623687 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:35:09.623904 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:35:09.624919 (+ 1015us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 09:35:09.624941 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:35:09.626908 (+ 1967us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 09:35:09.629164 (+ 2256us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:35:09.629172 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:35:09.629190 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:35:09.629573 (+ 383us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:35:09.629938 (+ 365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:35:09.629941 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:35:09.629943 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:35:09.630160 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:35:09.630765 (+ 605us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 09:35:09.630772 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 09:35:09.631081 (+ 309us) client_negotiation.cc:769] Sending connection context
0809 09:35:09.631995 (+ 914us) client_negotiation.cc:240] Negotiation successful
0809 09:35:09.632381 (+ 386us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":372,"thread_start_us":159,"threads_started":1}
I20260809 09:35:09.635136 8184 heartbeater.cc:347] Connected to a master server at 127.20.125.190:42791
I20260809 09:35:09.635452 8184 heartbeater.cc:464] Registering TS with master...
I20260809 09:35:09.636122 8184 heartbeater.cc:511] Master 127.20.125.190:42791 requested a full tablet report, sending...
I20260809 09:35:09.637682 7520 ts_manager.cc:194] Registered new tserver with Master: 9da8b5f247aa45c4a622ee3ba870c322 (127.20.125.131:34399)
I20260809 09:35:09.639436 7520 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.125.131@KRBTEST.COM'} at 127.20.125.131:58561
I20260809 09:35:09.642294 8199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:35:09.621462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53546 (local address 127.20.125.190:42791)
0809 09:35:09.630333 (+ 8871us) server_negotiation.cc:206] Beginning negotiation
0809 09:35:09.630342 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 09:35:09.630356 (+ 14us) server_negotiation.cc:407] Connection header received
0809 09:35:09.630394 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:35:09.630397 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:35:09.630445 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:35:09.630557 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:35:09.632338 (+ 1781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:35:09.633194 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:35:09.635145 (+ 1951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:35:09.635525 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:35:09.635631 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:35:09.637623 (+ 1992us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:35:09.637627 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:35:09.637630 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:35:09.637661 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:35:09.639667 (+ 2006us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:35:09.640196 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:35:09.640200 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:35:09.640202 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:35:09.640312 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:35:09.640652 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:35:09.640656 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:35:09.640658 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:35:09.640869 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:35:09.641047 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0809 09:35:09.641850 (+ 803us) server_negotiation.cc:299] Negotiation successful
0809 09:35:09.642024 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8789,"thread_start_us":113,"threads_started":1}
I20260809 09:35:09.981845 7795 heartbeater.cc:464] Registering TS with master...
I20260809 09:35:09.982169 7795 heartbeater.cc:511] Master 127.20.125.190:42791 requested a full tablet report, sending...
I20260809 09:35:09.983062 7519 ts_manager.cc:194] Registered new tserver with Master: 3003436f17b94848a8b90679923b7fa2 (127.20.125.129:39843)
I20260809 09:35:09.983981 7519 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.125.129@KRBTEST.COM'} at 127.20.125.129:54151
Aug 09 09:35:10 dist-test-slave-7kzw krb5kdc[6076](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786268110, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 09:35:10.296908 7996 heartbeater.cc:464] Registering TS with master...
I20260809 09:35:10.297217 7996 heartbeater.cc:511] Master 127.20.125.190:42791 requested a full tablet report, sending...
I20260809 09:35:10.298069 7519 ts_manager.cc:194] Registered new tserver with Master: eaa1c788ae2249848553125ad80445ca (127.20.125.130:35377)
I20260809 09:35:10.298784 7519 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.125.130@KRBTEST.COM'} at 127.20.125.130:39879
I20260809 09:35:10.308517 20982 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 09:35:10.321635 8214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:35:10.310940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53550 (local address 127.20.125.190:42791)
0809 09:35:10.311241 (+ 301us) server_negotiation.cc:206] Beginning negotiation
0809 09:35:10.311246 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 09:35:10.311694 (+ 448us) server_negotiation.cc:407] Connection header received
0809 09:35:10.311920 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:35:10.311923 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:35:10.311983 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:35:10.312106 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:35:10.312999 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:35:10.313972 (+ 973us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:35:10.315222 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:35:10.315602 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:35:10.315735 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:35:10.317548 (+ 1813us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:35:10.317552 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:35:10.317556 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:35:10.317584 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:35:10.319583 (+ 1999us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:35:10.320103 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:35:10.320107 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:35:10.320109 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:35:10.320221 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:35:10.320546 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:35:10.320550 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:35:10.320552 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:35:10.320779 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:35:10.320891 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0809 09:35:10.321281 (+ 390us) server_negotiation.cc:299] Negotiation successful
0809 09:35:10.321422 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":196,"thread_start_us":100,"threads_started":1}
I20260809 09:35:10.327183 7519 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53550:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 09:35:10.332581 7519 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 09:35:10.360255 8225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:35:10.350340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.131:34399 (local address 127.0.0.1:48782)
0809 09:35:10.352792 (+ 2452us) negotiation.cc:107] Waiting for socket to connect
0809 09:35:10.352848 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0809 09:35:10.352984 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:35:10.354160 (+ 1176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:35:10.354167 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:35:10.354194 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 09:35:10.354370 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:35:10.354378 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:35:10.356057 (+ 1679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:35:10.356062 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:35:10.357928 (+ 1866us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:35:10.357944 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:35:10.359514 (+ 1570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:35:10.359518 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:35:10.359646 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:35:10.359695 (+ 49us) client_negotiation.cc:769] Sending connection context
0809 09:35:10.359820 (+ 125us) client_negotiation.cc:240] Negotiation successful
0809 09:35:10.359994 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2258,"thread_start_us":155,"threads_started":1}
I20260809 09:35:10.360944 8222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:35:10.350495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48782 (local address 127.20.125.131:34399)
0809 09:35:10.353481 (+ 2986us) server_negotiation.cc:206] Beginning negotiation
0809 09:35:10.353488 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 09:35:10.353515 (+ 27us) server_negotiation.cc:407] Connection header received
0809 09:35:10.353602 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:35:10.353608 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:35:10.353970 (+ 362us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:35:10.354568 (+ 598us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 09:35:10.354714 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:35:10.355899 (+ 1185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:35:10.358125 (+ 2226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:35:10.359133 (+ 1008us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:35:10.359274 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:35:10.359429 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0809 09:35:10.360659 (+ 1230us) server_negotiation.cc:299] Negotiation successful
0809 09:35:10.360780 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2878,"thread_start_us":72,"threads_started":1}
I20260809 09:35:10.363867 8227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:35:10.352497 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41932 (local address 127.20.125.129:39843)
0809 09:35:10.353296 (+ 799us) server_negotiation.cc:206] Beginning negotiation
0809 09:35:10.353303 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 09:35:10.357266 (+ 3963us) server_negotiation.cc:407] Connection header received
0809 09:35:10.357515 (+ 249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:35:10.357521 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:35:10.357733 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:35:10.357862 (+ 129us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 09:35:10.358363 (+ 501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:35:10.360455 (+ 2092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:35:10.361993 (+ 1538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:35:10.362918 (+ 925us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:35:10.363020 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:35:10.363142 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0809 09:35:10.363540 (+ 398us) server_negotiation.cc:299] Negotiation successful
0809 09:35:10.363646 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":672,"thread_start_us":122,"threads_started":1}
I20260809 09:35:10.364310 8223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:35:10.352311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.129:39843 (local address 127.0.0.1:41932)
0809 09:35:10.357039 (+ 4728us) negotiation.cc:107] Waiting for socket to connect
0809 09:35:10.357054 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 09:35:10.357350 (+ 296us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:35:10.358081 (+ 731us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:35:10.358084 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:35:10.358102 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 09:35:10.358216 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:35:10.358223 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:35:10.360612 (+ 2389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:35:10.360615 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:35:10.361834 (+ 1219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:35:10.361847 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:35:10.363258 (+ 1411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:35:10.363261 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:35:10.363346 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:35:10.363396 (+ 50us) client_negotiation.cc:769] Sending connection context
0809 09:35:10.364076 (+ 680us) client_negotiation.cc:240] Negotiation successful
0809 09:35:10.364156 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4661,"thread_start_us":92,"threads_started":1}
I20260809 09:35:10.365172 8087 tablet_service.cc:1467] Processing CreateTablet for tablet e8e937980590474993d4f5a6e5abd471 (DEFAULT_TABLE table=test-table [id=2027f477ff2b40a5aa396729c3fe5eaf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 09:35:10.366426 8087 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e8e937980590474993d4f5a6e5abd471. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 09:35:10.367810 7710 tablet_service.cc:1467] Processing CreateTablet for tablet e8e937980590474993d4f5a6e5abd471 (DEFAULT_TABLE table=test-table [id=2027f477ff2b40a5aa396729c3fe5eaf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 09:35:10.369033 7710 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e8e937980590474993d4f5a6e5abd471. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 09:35:10.371624 8221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:35:10.351622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.130:35377 (local address 127.0.0.1:56804)
0809 09:35:10.356151 (+ 4529us) negotiation.cc:107] Waiting for socket to connect
0809 09:35:10.356167 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 09:35:10.356274 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:35:10.362118 (+ 5844us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:35:10.362121 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:35:10.362144 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 09:35:10.362238 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:35:10.362244 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:35:10.368506 (+ 6262us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:35:10.368510 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:35:10.369790 (+ 1280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:35:10.369804 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:35:10.371137 (+ 1333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:35:10.371141 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:35:10.371262 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:35:10.371290 (+ 28us) client_negotiation.cc:769] Sending connection context
0809 09:35:10.371352 (+ 62us) client_negotiation.cc:240] Negotiation successful
0809 09:35:10.371438 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4461,"thread_start_us":105,"threads_started":1}
I20260809 09:35:10.372409 8224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:35:10.351782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56804 (local address 127.20.125.130:35377)
0809 09:35:10.359671 (+ 7889us) server_negotiation.cc:206] Beginning negotiation
0809 09:35:10.359679 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 09:35:10.359706 (+ 27us) server_negotiation.cc:407] Connection header received
0809 09:35:10.359793 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:35:10.359800 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:35:10.360030 (+ 230us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:35:10.362405 (+ 2375us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 09:35:10.362602 (+ 197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:35:10.363458 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:35:10.369988 (+ 6530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:35:10.370985 (+ 997us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:35:10.371955 (+ 970us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:35:10.372100 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0809 09:35:10.372156 (+ 56us) server_negotiation.cc:299] Negotiation successful
0809 09:35:10.372252 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7766,"thread_start_us":95,"threads_started":1}
I20260809 09:35:10.375389 7898 tablet_service.cc:1467] Processing CreateTablet for tablet e8e937980590474993d4f5a6e5abd471 (DEFAULT_TABLE table=test-table [id=2027f477ff2b40a5aa396729c3fe5eaf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 09:35:10.376471 7898 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e8e937980590474993d4f5a6e5abd471. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 09:35:10.385635 8230 tablet_bootstrap.cc:492] T e8e937980590474993d4f5a6e5abd471 P 9da8b5f247aa45c4a622ee3ba870c322: Bootstrap starting.
I20260809 09:35:10.388530 8230 tablet_bootstrap.cc:654] T e8e937980590474993d4f5a6e5abd471 P 9da8b5f247aa45c4a622ee3ba870c322: Neither blocks nor log segments found. Creating new log.
I20260809 09:35:10.389532 8231 tablet_bootstrap.cc:492] T e8e937980590474993d4f5a6e5abd471 P eaa1c788ae2249848553125ad80445ca: Bootstrap starting.
I20260809 09:35:10.389684 8230 log.cc:824] T e8e937980590474993d4f5a6e5abd471 P 9da8b5f247aa45c4a622ee3ba870c322: Log is configured to *not* fsync() on all Append() calls
I20260809 09:35:10.391897 8231 tablet_bootstrap.cc:654] T e8e937980590474993d4f5a6e5abd471 P eaa1c788ae2249848553125ad80445ca: Neither blocks nor log segments found. Creating new log.
I20260809 09:35:10.392338 8228 tablet_bootstrap.cc:492] T e8e937980590474993d4f5a6e5abd471 P 3003436f17b94848a8b90679923b7fa2: Bootstrap starting.
I20260809 09:35:10.392434 8230 tablet_bootstrap.cc:492] T e8e937980590474993d4f5a6e5abd471 P 9da8b5f247aa45c4a622ee3ba870c322: No bootstrap required, opened a new log
I20260809 09:35:10.392656 8230 ts_tablet_manager.cc:1397] T e8e937980590474993d4f5a6e5abd471 P 9da8b5f247aa45c4a622ee3ba870c322: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260809 09:35:10.393247 8231 log.cc:824] T e8e937980590474993d4f5a6e5abd471 P eaa1c788ae2249848553125ad80445ca: Log is configured to *not* fsync() on all Append() calls
I20260809 09:35:10.395813 8228 tablet_bootstrap.cc:654] T e8e937980590474993d4f5a6e5abd471 P 3003436f17b94848a8b90679923b7fa2: Neither blocks nor log segments found. Creating new log.
I20260809 09:35:10.396814 8228 log.cc:824] T e8e937980590474993d4f5a6e5abd471 P 3003436f17b94848a8b90679923b7fa2: Log is configured to *not* fsync() on all Append() calls
I20260809 09:35:10.398998 8228 tablet_bootstrap.cc:492] T e8e937980590474993d4f5a6e5abd471 P 3003436f17b94848a8b90679923b7fa2: No bootstrap required, opened a new log
I20260809 09:35:10.399185 8228 ts_tablet_manager.cc:1397] T e8e937980590474993d4f5a6e5abd471 P 3003436f17b94848a8b90679923b7fa2: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260809 09:35:10.398948 8230 raft_consensus.cc:348] T e8e937980590474993d4f5a6e5abd471 P 9da8b5f247aa45c4a622ee3ba870c322 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9da8b5f247aa45c4a622ee3ba870c322" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 34399 } } peers { permanent_uuid: "eaa1c788ae2249848553125ad80445ca" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 35377 } } peers { permanent_uuid: "3003436f17b94848a8b90679923b7fa2" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 39843 } }
I20260809 09:35:10.399281 8230 raft_consensus.cc:374] T e8e937980590474993d4f5a6e5abd471 P 9da8b5f247aa45c4a622ee3ba870c322 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 09:35:10.399353 8230 raft_consensus.cc:729] T e8e937980590474993d4f5a6e5abd471 P 9da8b5f247aa45c4a622ee3ba870c322 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9da8b5f247aa45c4a622ee3ba870c322, State: Initialized, Role: FOLLOWER
I20260809 09:35:10.400020 8230 consensus_queue.cc:260] T e8e937980590474993d4f5a6e5abd471 P 9da8b5f247aa45c4a622ee3ba870c322 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9da8b5f247aa45c4a622ee3ba870c322" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 34399 } } peers { permanent_uuid: "eaa1c788ae2249848553125ad80445ca" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 35377 } } peers { permanent_uuid: "3003436f17b94848a8b90679923b7fa2" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 39843 } }
I20260809 09:35:10.401227 8230 ts_tablet_manager.cc:1428] T e8e937980590474993d4f5a6e5abd471 P 9da8b5f247aa45c4a622ee3ba870c322: Time spent starting tablet: real 0.009s user 0.006s sys 0.000s
I20260809 09:35:10.401651 8184 heartbeater.cc:503] Master 127.20.125.190:42791 was elected leader, sending a full tablet report...
I20260809 09:35:10.403287 8228 raft_consensus.cc:348] T e8e937980590474993d4f5a6e5abd471 P 3003436f17b94848a8b90679923b7fa2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9da8b5f247aa45c4a622ee3ba870c322" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 34399 } } peers { permanent_uuid: "eaa1c788ae2249848553125ad80445ca" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 35377 } } peers { permanent_uuid: "3003436f17b94848a8b90679923b7fa2" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 39843 } }
I20260809 09:35:10.403530 8228 raft_consensus.cc:374] T e8e937980590474993d4f5a6e5abd471 P 3003436f17b94848a8b90679923b7fa2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 09:35:10.403571 8228 raft_consensus.cc:729] T e8e937980590474993d4f5a6e5abd471 P 3003436f17b94848a8b90679923b7fa2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3003436f17b94848a8b90679923b7fa2, State: Initialized, Role: FOLLOWER
I20260809 09:35:10.404305 8228 consensus_queue.cc:260] T e8e937980590474993d4f5a6e5abd471 P 3003436f17b94848a8b90679923b7fa2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9da8b5f247aa45c4a622ee3ba870c322" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 34399 } } peers { permanent_uuid: "eaa1c788ae2249848553125ad80445ca" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 35377 } } peers { permanent_uuid: "3003436f17b94848a8b90679923b7fa2" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 39843 } }
I20260809 09:35:10.405504 8228 ts_tablet_manager.cc:1428] T e8e937980590474993d4f5a6e5abd471 P 3003436f17b94848a8b90679923b7fa2: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260809 09:35:10.405892 7795 heartbeater.cc:503] Master 127.20.125.190:42791 was elected leader, sending a full tablet report...
I20260809 09:35:10.412318 8231 tablet_bootstrap.cc:492] T e8e937980590474993d4f5a6e5abd471 P eaa1c788ae2249848553125ad80445ca: No bootstrap required, opened a new log
I20260809 09:35:10.412537 8231 ts_tablet_manager.cc:1397] T e8e937980590474993d4f5a6e5abd471 P eaa1c788ae2249848553125ad80445ca: Time spent bootstrapping tablet: real 0.023s user 0.006s sys 0.000s
I20260809 09:35:10.416594 8231 raft_consensus.cc:348] T e8e937980590474993d4f5a6e5abd471 P eaa1c788ae2249848553125ad80445ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9da8b5f247aa45c4a622ee3ba870c322" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 34399 } } peers { permanent_uuid: "eaa1c788ae2249848553125ad80445ca" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 35377 } } peers { permanent_uuid: "3003436f17b94848a8b90679923b7fa2" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 39843 } }
I20260809 09:35:10.416903 8231 raft_consensus.cc:374] T e8e937980590474993d4f5a6e5abd471 P eaa1c788ae2249848553125ad80445ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 09:35:10.416990 8231 raft_consensus.cc:729] T e8e937980590474993d4f5a6e5abd471 P eaa1c788ae2249848553125ad80445ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eaa1c788ae2249848553125ad80445ca, State: Initialized, Role: FOLLOWER
I20260809 09:35:10.417487 8231 consensus_queue.cc:260] T e8e937980590474993d4f5a6e5abd471 P eaa1c788ae2249848553125ad80445ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9da8b5f247aa45c4a622ee3ba870c322" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 34399 } } peers { permanent_uuid: "eaa1c788ae2249848553125ad80445ca" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 35377 } } peers { permanent_uuid: "3003436f17b94848a8b90679923b7fa2" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 39843 } }
I20260809 09:35:10.418300 7996 heartbeater.cc:503] Master 127.20.125.190:42791 was elected leader, sending a full tablet report...
I20260809 09:35:10.418618 8231 ts_tablet_manager.cc:1428] T e8e937980590474993d4f5a6e5abd471 P eaa1c788ae2249848553125ad80445ca: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
W20260809 09:35:10.437559 7796 tablet.cc:2369] T e8e937980590474993d4f5a6e5abd471 P 3003436f17b94848a8b90679923b7fa2: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 09 09:35:10 dist-test-slave-7kzw krb5kdc[6076](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268110, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.190@KRBTEST.COM for HTTP/127.20.125.148@KRBTEST.COM
W20260809 09:35:10.514674 7997 tablet.cc:2369] T e8e937980590474993d4f5a6e5abd471 P eaa1c788ae2249848553125ad80445ca: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 09:35:10.562650 8239 raft_consensus.cc:482] T e8e937980590474993d4f5a6e5abd471 P 9da8b5f247aa45c4a622ee3ba870c322 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 09:35:10.562901 8239 raft_consensus.cc:504] T e8e937980590474993d4f5a6e5abd471 P 9da8b5f247aa45c4a622ee3ba870c322 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9da8b5f247aa45c4a622ee3ba870c322" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 34399 } } peers { permanent_uuid: "eaa1c788ae2249848553125ad80445ca" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 35377 } } peers { permanent_uuid: "3003436f17b94848a8b90679923b7fa2" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 39843 } }
I20260809 09:35:10.564428 8239 leader_election.cc:288] T e8e937980590474993d4f5a6e5abd471 P 9da8b5f247aa45c4a622ee3ba870c322 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eaa1c788ae2249848553125ad80445ca (127.20.125.130:35377), 3003436f17b94848a8b90679923b7fa2 (127.20.125.129:39843)
I20260809 09:35:10.571753 8227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:35:10.565488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.131:54425 (local address 127.20.125.129:39843)
0809 09:35:10.565660 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0809 09:35:10.565666 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 09:35:10.566619 (+ 953us) server_negotiation.cc:407] Connection header received
0809 09:35:10.566843 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:35:10.566847 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:35:10.566897 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:35:10.566948 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 09:35:10.567534 (+ 586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:35:10.568364 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:35:10.570286 (+ 1922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:35:10.570924 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:35:10.571027 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:35:10.571091 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0809 09:35:10.571526 (+ 435us) server_negotiation.cc:299] Negotiation successful
0809 09:35:10.571608 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260809 09:35:10.572456 8242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:35:10.564895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.129:39843 (local address 127.20.125.131:54425)
0809 09:35:10.566530 (+ 1635us) negotiation.cc:107] Waiting for socket to connect
0809 09:35:10.566547 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 09:35:10.566709 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:35:10.567134 (+ 425us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:35:10.567137 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:35:10.567156 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 09:35:10.567390 (+ 234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:35:10.567398 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:35:10.568523 (+ 1125us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:35:10.568526 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:35:10.570141 (+ 1615us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:35:10.570153 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:35:10.571216 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:35:10.571220 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:35:10.571369 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:35:10.571394 (+ 25us) client_negotiation.cc:769] Sending connection context
0809 09:35:10.572196 (+ 802us) client_negotiation.cc:240] Negotiation successful
0809 09:35:10.572293 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1511,"thread_start_us":141,"threads_started":1}
I20260809 09:35:10.573318 7736 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8e937980590474993d4f5a6e5abd471" candidate_uuid: "9da8b5f247aa45c4a622ee3ba870c322" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3003436f17b94848a8b90679923b7fa2" is_pre_election: true
I20260809 09:35:10.573722 7736 raft_consensus.cc:2454] T e8e937980590474993d4f5a6e5abd471 P 3003436f17b94848a8b90679923b7fa2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9da8b5f247aa45c4a622ee3ba870c322 in term 0.
I20260809 09:35:10.574775 8022 leader_election.cc:302] T e8e937980590474993d4f5a6e5abd471 P 9da8b5f247aa45c4a622ee3ba870c322 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3003436f17b94848a8b90679923b7fa2, 9da8b5f247aa45c4a622ee3ba870c322; no voters:
I20260809 09:35:10.575141 8239 raft_consensus.cc:2781] T e8e937980590474993d4f5a6e5abd471 P 9da8b5f247aa45c4a622ee3ba870c322 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 09:35:10.575222 8239 raft_consensus.cc:482] T e8e937980590474993d4f5a6e5abd471 P 9da8b5f247aa45c4a622ee3ba870c322 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 09:35:10.575258 8239 raft_consensus.cc:3037] T e8e937980590474993d4f5a6e5abd471 P 9da8b5f247aa45c4a622ee3ba870c322 [term 0 FOLLOWER]: Advancing to term 1
I20260809 09:35:10.577059 8239 raft_consensus.cc:504] T e8e937980590474993d4f5a6e5abd471 P 9da8b5f247aa45c4a622ee3ba870c322 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9da8b5f247aa45c4a622ee3ba870c322" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 34399 } } peers { permanent_uuid: "eaa1c788ae2249848553125ad80445ca" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 35377 } } peers { permanent_uuid: "3003436f17b94848a8b90679923b7fa2" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 39843 } }
I20260809 09:35:10.577503 8239 leader_election.cc:288] T e8e937980590474993d4f5a6e5abd471 P 9da8b5f247aa45c4a622ee3ba870c322 [CANDIDATE]: Term 1 election: Requested vote from peers eaa1c788ae2249848553125ad80445ca (127.20.125.130:35377), 3003436f17b94848a8b90679923b7fa2 (127.20.125.129:39843)
I20260809 09:35:10.578300 7736 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8e937980590474993d4f5a6e5abd471" candidate_uuid: "9da8b5f247aa45c4a622ee3ba870c322" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3003436f17b94848a8b90679923b7fa2"
I20260809 09:35:10.578436 7736 raft_consensus.cc:3037] T e8e937980590474993d4f5a6e5abd471 P 3003436f17b94848a8b90679923b7fa2 [term 0 FOLLOWER]: Advancing to term 1
I20260809 09:35:10.580322 7736 raft_consensus.cc:2454] T e8e937980590474993d4f5a6e5abd471 P 3003436f17b94848a8b90679923b7fa2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9da8b5f247aa45c4a622ee3ba870c322 in term 1.
I20260809 09:35:10.580886 8022 leader_election.cc:302] T e8e937980590474993d4f5a6e5abd471 P 9da8b5f247aa45c4a622ee3ba870c322 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3003436f17b94848a8b90679923b7fa2, 9da8b5f247aa45c4a622ee3ba870c322; no voters:
I20260809 09:35:10.581549 8239 raft_consensus.cc:2781] T e8e937980590474993d4f5a6e5abd471 P 9da8b5f247aa45c4a622ee3ba870c322 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 09:35:10.581887 8239 raft_consensus.cc:686] T e8e937980590474993d4f5a6e5abd471 P 9da8b5f247aa45c4a622ee3ba870c322 [term 1 LEADER]: Becoming Leader. State: Replica: 9da8b5f247aa45c4a622ee3ba870c322, State: Running, Role: LEADER
I20260809 09:35:10.582336 8239 consensus_queue.cc:237] T e8e937980590474993d4f5a6e5abd471 P 9da8b5f247aa45c4a622ee3ba870c322 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9da8b5f247aa45c4a622ee3ba870c322" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 34399 } } peers { permanent_uuid: "eaa1c788ae2249848553125ad80445ca" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 35377 } } peers { permanent_uuid: "3003436f17b94848a8b90679923b7fa2" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 39843 } }
I20260809 09:35:10.586799 7520 catalog_manager.cc:5495] T e8e937980590474993d4f5a6e5abd471 P 9da8b5f247aa45c4a622ee3ba870c322 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9da8b5f247aa45c4a622ee3ba870c322 (127.20.125.131). New cstate: current_term: 1 leader_uuid: "9da8b5f247aa45c4a622ee3ba870c322" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9da8b5f247aa45c4a622ee3ba870c322" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 34399 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eaa1c788ae2249848553125ad80445ca" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 35377 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3003436f17b94848a8b90679923b7fa2" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 39843 } health_report { overall_health: UNKNOWN } } }
I20260809 09:35:10.598740 8224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:35:10.566245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.131:47683 (local address 127.20.125.130:35377)
0809 09:35:10.566368 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0809 09:35:10.566372 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 09:35:10.573195 (+ 6823us) server_negotiation.cc:407] Connection header received
0809 09:35:10.573401 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:35:10.573405 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:35:10.573462 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:35:10.573513 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 09:35:10.573895 (+ 382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:35:10.581360 (+ 7465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:35:10.591512 (+ 10152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:35:10.592183 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:35:10.592293 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:35:10.592349 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0809 09:35:10.598480 (+ 6131us) server_negotiation.cc:299] Negotiation successful
0809 09:35:10.598596 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260809 09:35:10.599519 8242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:35:10.565760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.130:35377 (local address 127.20.125.131:47683)
0809 09:35:10.573113 (+ 7353us) negotiation.cc:107] Waiting for socket to connect
0809 09:35:10.573128 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 09:35:10.573270 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:35:10.573659 (+ 389us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:35:10.573662 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:35:10.573678 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 09:35:10.573755 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:35:10.573762 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:35:10.588273 (+ 14511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:35:10.588276 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:35:10.591323 (+ 3047us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:35:10.591338 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:35:10.598147 (+ 6809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:35:10.598151 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:35:10.598258 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:35:10.598287 (+ 29us) client_negotiation.cc:769] Sending connection context
0809 09:35:10.599275 (+ 988us) client_negotiation.cc:240] Negotiation successful
0809 09:35:10.599375 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7295,"thread_start_us":172,"threads_started":1}
I20260809 09:35:10.600440 7910 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8e937980590474993d4f5a6e5abd471" candidate_uuid: "9da8b5f247aa45c4a622ee3ba870c322" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eaa1c788ae2249848553125ad80445ca" is_pre_election: true
I20260809 09:35:10.600812 7910 raft_consensus.cc:2454] T e8e937980590474993d4f5a6e5abd471 P eaa1c788ae2249848553125ad80445ca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9da8b5f247aa45c4a622ee3ba870c322 in term 0.
I20260809 09:35:10.601157 7911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8e937980590474993d4f5a6e5abd471" candidate_uuid: "9da8b5f247aa45c4a622ee3ba870c322" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eaa1c788ae2249848553125ad80445ca"
I20260809 09:35:10.601281 7911 raft_consensus.cc:3037] T e8e937980590474993d4f5a6e5abd471 P eaa1c788ae2249848553125ad80445ca [term 0 FOLLOWER]: Advancing to term 1
I20260809 09:35:10.602835 7911 raft_consensus.cc:2454] T e8e937980590474993d4f5a6e5abd471 P eaa1c788ae2249848553125ad80445ca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9da8b5f247aa45c4a622ee3ba870c322 in term 1.
I20260809 09:35:10.607481 8222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:35:10.602428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48798 (local address 127.20.125.131:34399)
0809 09:35:10.602552 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0809 09:35:10.602556 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 09:35:10.602599 (+ 43us) server_negotiation.cc:407] Connection header received
0809 09:35:10.602671 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:35:10.602674 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:35:10.602735 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:35:10.602832 (+ 97us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 09:35:10.603274 (+ 442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:35:10.604028 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:35:10.605305 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:35:10.605643 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:35:10.605744 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:35:10.606174 (+ 430us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 09:35:10.606734 (+ 560us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 09:35:10.606848 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0809 09:35:10.607295 (+ 447us) server_negotiation.cc:299] Negotiation successful
0809 09:35:10.607354 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 09:35:10.619837 7736 raft_consensus.cc:1261] T e8e937980590474993d4f5a6e5abd471 P 3003436f17b94848a8b90679923b7fa2 [term 1 FOLLOWER]: Refusing update from remote peer 9da8b5f247aa45c4a622ee3ba870c322: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 09:35:10.620826 7911 raft_consensus.cc:1261] T e8e937980590474993d4f5a6e5abd471 P eaa1c788ae2249848553125ad80445ca [term 1 FOLLOWER]: Refusing update from remote peer 9da8b5f247aa45c4a622ee3ba870c322: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260809 09:35:10.621380 8185 tablet.cc:2369] T e8e937980590474993d4f5a6e5abd471 P 9da8b5f247aa45c4a622ee3ba870c322: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 09:35:10.622347 8239 consensus_queue.cc:1035] T e8e937980590474993d4f5a6e5abd471 P 9da8b5f247aa45c4a622ee3ba870c322 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eaa1c788ae2249848553125ad80445ca" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 35377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 09:35:10.622852 8239 consensus_queue.cc:1035] T e8e937980590474993d4f5a6e5abd471 P 9da8b5f247aa45c4a622ee3ba870c322 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3003436f17b94848a8b90679923b7fa2" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 39843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 09:35:10.637231 8248 mvcc.cc:205] Tried to move back new op lower bound from 7316554181070450688 to 7316554180950478848. Current Snapshot: MvccSnapshot[applied={T|T < 7316554181070450688}]
I20260809 09:35:10.638603 8255 mvcc.cc:205] Tried to move back new op lower bound from 7316554181070450688 to 7316554180950478848. Current Snapshot: MvccSnapshot[applied={T|T < 7316554181070450688}]
I20260809 09:35:10.642735 8253 mvcc.cc:205] Tried to move back new op lower bound from 7316554181070450688 to 7316554180950478848. Current Snapshot: MvccSnapshot[applied={T|T < 7316554181070450688}]
I20260809 09:35:10.656442 7520 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53550:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 09:35:10.656652 7520 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53550:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 09:35:10.659796 7520 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5c09265aec1a48b894b76b40aa1fc8b7: Sending DeleteTablet for 3 replicas of tablet e8e937980590474993d4f5a6e5abd471
I20260809 09:35:10.660971 7898 tablet_service.cc:1514] Processing DeleteTablet for tablet e8e937980590474993d4f5a6e5abd471 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 09:35:10 UTC) from {username='kudu', principal='kudu/127.20.125.190@KRBTEST.COM'} at 127.0.0.1:56804
I20260809 09:35:10.661942 7710 tablet_service.cc:1514] Processing DeleteTablet for tablet e8e937980590474993d4f5a6e5abd471 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 09:35:10 UTC) from {username='kudu', principal='kudu/127.20.125.190@KRBTEST.COM'} at 127.0.0.1:41932
I20260809 09:35:10.662789 8087 tablet_service.cc:1514] Processing DeleteTablet for tablet e8e937980590474993d4f5a6e5abd471 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 09:35:10 UTC) from {username='kudu', principal='kudu/127.20.125.190@KRBTEST.COM'} at 127.0.0.1:48782
I20260809 09:35:10.666079 8266 tablet_replica.cc:330] T e8e937980590474993d4f5a6e5abd471 P 3003436f17b94848a8b90679923b7fa2: stopping tablet replica
I20260809 09:35:10.666764 8267 tablet_replica.cc:330] T e8e937980590474993d4f5a6e5abd471 P 9da8b5f247aa45c4a622ee3ba870c322: stopping tablet replica
I20260809 09:35:10.667104 8267 raft_consensus.cc:2229] T e8e937980590474993d4f5a6e5abd471 P 9da8b5f247aa45c4a622ee3ba870c322 [term 1 LEADER]: Raft consensus shutting down.
I20260809 09:35:10.667312 20982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu with pid 7578
I20260809 09:35:10.673972 8265 tablet_replica.cc:330] T e8e937980590474993d4f5a6e5abd471 P eaa1c788ae2249848553125ad80445ca: stopping tablet replica
I20260809 09:35:10.674345 8265 raft_consensus.cc:2229] T e8e937980590474993d4f5a6e5abd471 P eaa1c788ae2249848553125ad80445ca [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 09:35:10.674764 8265 raft_consensus.cc:2258] T e8e937980590474993d4f5a6e5abd471 P eaa1c788ae2249848553125ad80445ca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 09:35:10.682789 8267 raft_consensus.cc:2258] T e8e937980590474993d4f5a6e5abd471 P 9da8b5f247aa45c4a622ee3ba870c322 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 09:35:10.687865 20982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu with pid 7799
W20260809 09:35:10.688359 7500 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.20.125.129:39843 (error 108)
I20260809 09:35:10.708465 8267 ts_tablet_manager.cc:1905] T e8e937980590474993d4f5a6e5abd471 P 9da8b5f247aa45c4a622ee3ba870c322: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 09:35:10.711908 8267 ts_tablet_manager.cc:1918] T e8e937980590474993d4f5a6e5abd471 P 9da8b5f247aa45c4a622ee3ba870c322: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 09:35:10.712165 8267 log.cc:1196] T e8e937980590474993d4f5a6e5abd471 P 9da8b5f247aa45c4a622ee3ba870c322: Deleting WAL directory at /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786267941944493-20982-0/minicluster-data/ts-2/wal/wals/e8e937980590474993d4f5a6e5abd471
I20260809 09:35:10.712723 8267 ts_tablet_manager.cc:1939] T e8e937980590474993d4f5a6e5abd471 P 9da8b5f247aa45c4a622ee3ba870c322: Deleting consensus metadata
I20260809 09:35:10.714775 7503 catalog_manager.cc:4841] TS 9da8b5f247aa45c4a622ee3ba870c322 (127.20.125.131:34399): tablet e8e937980590474993d4f5a6e5abd471 (table test-table [id=2027f477ff2b40a5aa396729c3fe5eaf]) successfully deleted
I20260809 09:35:10.716542 20982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu with pid 8000
I20260809 09:35:10.742300 8221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:35:10.741802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.129:39843 (local address 127.0.0.1:41934)
0809 09:35:10.742055 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0809 09:35:10.742159 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.129:39843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143}
W20260809 09:35:10.743086 7500 catalog_manager.cc:4584] TS 3003436f17b94848a8b90679923b7fa2 (127.20.125.129:39843): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e8e937980590474993d4f5a6e5abd471: Network error: Client connection negotiation failed: client connection to 127.20.125.129:39843: connect: Connection refused (error 111)
I20260809 09:35:10.743599 8221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:35:10.742868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.130:35377 (local address 127.0.0.1:56816)
0809 09:35:10.743425 (+ 557us) negotiation.cc:107] Waiting for socket to connect
0809 09:35:10.743493 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.130:35377: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":463}
I20260809 09:35:10.748646 20982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu with pid 7474
I20260809 09:35:10.765771 20982 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260809 09:35:11.111691 20982 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260809 09:35:11.111819 20982 mini_ranger.cc:66] Stopping Ranger...
I20260809 09:35:11.467624 20982 mini_ranger.cc:68] Stopped Ranger
2026-08-09 09:35:11.467 UTC [6120] LOG: received smart shutdown request
2026-08-09 09:35:11.472 UTC [6120] LOG: background worker "logical replication launcher" (PID 6132) exited with exit code 1
2026-08-09 09:35:11.472 UTC [6127] LOG: shutting down
2026-08-09 09:35:11.650 UTC [6120] LOG: database system is shut down
2026-08-09T09:35:11Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (50861 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786267941944493-20982-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 09:35:11 dist-test-slave-7kzw krb5kdc[8282](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 09:35:11 dist-test-slave-7kzw krb5kdc[8282](info): set up 2 sockets
Aug 09 09:35:11 dist-test-slave-7kzw krb5kdc[8282](info): commencing operation
krb5kdc: starting...
W20260809 09:35:13.055838 20982 mini_kdc.cc:121] Time spent starting KDC: real 1.247s user 0.001s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786267941944493-20982-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786267941944493-20982-0/krb5kdc/test-user.keytab.
Aug 09 09:35:13 dist-test-slave-7kzw krb5kdc[8282](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268113, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T09:35:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T09:35:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.125.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.125.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.125.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.190.keytab.
Entry for principal kudu/127.20.125.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.190.keytab.
I20260809 09:35:13.189163 20982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
/tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786267941944493-20982-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786267941944493-20982-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786267941944493-20982-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786267941944493-20982-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.125.190:41293
--webserver_interface=127.20.125.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.190.keytab
--principal=kudu/127.20.125.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.125.148:45373
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.125.190:41293,127.20.125.189:35185,127.20.125.188:37597
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786267941944493-20982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786267941944493-20982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786267941944493-20982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 09:35:13.305102 8301 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 09:35:13.305414 8301 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 09:35:13.305459 8301 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 09:35:13.309582 8301 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 09:35:13.309659 8301 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 09:35:13.309680 8301 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 09:35:13.309697 8301 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 09:35:13.309715 8301 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 09:35:13.315573 8301 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.125.148:45373
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786267941944493-20982-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786267941944493-20982-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.125.190:41293,127.20.125.189:35185,127.20.125.188:37597
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.190.keytab
--principal=kudu/127.20.125.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.125.190:41293
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786267941944493-20982-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.125.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8301
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786267941944493-20982-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 08:38:55 UTC on ccycloud.quasar-njjleu.root.comops.site
build id 4575935
I20260809 09:35:13.316872 8301 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 09:35:13.318040 8301 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 09:35:13.327231 8301 server_base.cc:1034] running on GCE node
W20260809 09:35:13.327299 8310 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:35:13.327154 8309 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:35:13.327211 8313 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:35:13.328140 8301 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 09:35:13.329267 8301 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 09:35:13.330453 8301 hybrid_clock.cc:648] HybridClock initialized: now 1786268113330433 us; error 33 us; skew 500 ppm
Aug 09 09:35:13 dist-test-slave-7kzw krb5kdc[8282](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268113, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 09:35:13.333493 8301 init.cc:377] Logged in from keytab as kudu/127.20.125.190@KRBTEST.COM (short username kudu)
I20260809 09:35:13.334398 8301 webserver.cc:460] Webserver started at http://127.20.125.190:42967/ using document root <none> and password file <none>
I20260809 09:35:13.335028 8301 fs_manager.cc:359] Metadata directory not provided
I20260809 09:35:13.335098 8301 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 09:35:13.335327 8301 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 09:35:13.337430 8301 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786267941944493-20982-0/minicluster-data/master-0/data/instance:
uuid: "84d808263ac6480a9e1a73424b3da25e"
format_stamp: "Formatted at 2026-08-09 09:35:13 on dist-test-slave-7kzw"
server_key: "6be8708e0fb9bfd0e6a425db59a37523"
server_key_iv: "134ef1ca724e23ab4850de8adabd20ef"
server_key_version: "encryptionkey@0"
I20260809 09:35:13.338033 8301 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786267941944493-20982-0/minicluster-data/master-0/wal/instance:
uuid: "84d808263ac6480a9e1a73424b3da25e"
format_stamp: "Formatted at 2026-08-09 09:35:13 on dist-test-slave-7kzw"
server_key: "6be8708e0fb9bfd0e6a425db59a37523"
server_key_iv: "134ef1ca724e23ab4850de8adabd20ef"
server_key_version: "encryptionkey@0"
I20260809 09:35:13.342440 8301 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260809 09:35:13.345433 8326 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 09:35:13.346911 8301 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260809 09:35:13.347069 8301 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786267941944493-20982-0/minicluster-data/master-0/data,/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786267941944493-20982-0/minicluster-data/master-0/wal
uuid: "84d808263ac6480a9e1a73424b3da25e"
format_stamp: "Formatted at 2026-08-09 09:35:13 on dist-test-slave-7kzw"
server_key: "6be8708e0fb9bfd0e6a425db59a37523"
server_key_iv: "134ef1ca724e23ab4850de8adabd20ef"
server_key_version: "encryptionkey@0"
I20260809 09:35:13.347261 8301 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786267941944493-20982-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786267941944493-20982-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786267941944493-20982-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 09:35:13.366093 8301 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 09:35:13.369901 8301 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 09:35:13.370085 8301 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 09:35:13.378098 8301 rpc_server.cc:307] RPC server started. Bound to: 127.20.125.190:41293
I20260809 09:35:13.378121 8402 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.125.190:41293 every 8 connection(s)
I20260809 09:35:13.379223 8301 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786267941944493-20982-0/minicluster-data/master-0/data/info.pb
I20260809 09:35:13.382123 8403 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 09:35:13.385951 8403 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.125.190" port: 41293 } has no permanent_uuid. Determining permanent_uuid...
I20260809 09:35:13.386554 20982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu as pid 8301
I20260809 09:35:13.386706 20982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786267941944493-20982-0/minicluster-data/master-0/wal/instance
I20260809 09:35:13.386998 20982 external_mini_cluster.cc:1428] Setting key 41c25aa4259395facc8e0ff173895f09
WARNING: no policy specified for kudu/127.20.125.189@KRBTEST.COM; defaulting to no policy
Aug 09 09:35:13 dist-test-slave-7kzw krb5kdc[8282](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268113, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.190@KRBTEST.COM for kudu/127.20.125.190@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.125.189@KRBTEST.COM" created.
I20260809 09:35:13.403851 8405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:35:13.388660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:41293 (local address 127.0.0.1:35194)
0809 09:35:13.389282 (+ 622us) negotiation.cc:107] Waiting for socket to connect
0809 09:35:13.389359 (+ 77us) client_negotiation.cc:174] Beginning negotiation
0809 09:35:13.390661 (+ 1302us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:35:13.391427 (+ 766us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:35:13.391436 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:35:13.391973 (+ 537us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:35:13.392414 (+ 441us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:35:13.392443 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:35:13.393667 (+ 1224us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:35:13.393673 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:35:13.395050 (+ 1377us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:35:13.395064 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:35:13.395741 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:35:13.395748 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:35:13.395883 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:35:13.396554 (+ 671us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 09:35:13.396574 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:35:13.398313 (+ 1739us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 09:35:13.400655 (+ 2342us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:35:13.400664 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:35:13.400678 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:35:13.400906 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:35:13.401268 (+ 362us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:35:13.401273 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:35:13.401275 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:35:13.401388 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:35:13.402001 (+ 613us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 09:35:13.402007 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 09:35:13.402141 (+ 134us) client_negotiation.cc:769] Sending connection context
0809 09:35:13.402317 (+ 176us) client_negotiation.cc:240] Negotiation successful
0809 09:35:13.402753 (+ 436us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":351,"mutex_wait_us":21,"thread_start_us":160,"threads_started":1}
I20260809 09:35:13.403851 8409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:35:13.389661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35194 (local address 127.20.125.190:41293)
0809 09:35:13.389961 (+ 300us) server_negotiation.cc:206] Beginning negotiation
0809 09:35:13.389972 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 09:35:13.390011 (+ 39us) server_negotiation.cc:407] Connection header received
0809 09:35:13.391002 (+ 991us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:35:13.391006 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:35:13.391203 (+ 197us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:35:13.391365 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:35:13.392625 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:35:13.393517 (+ 892us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:35:13.395193 (+ 1676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:35:13.395568 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:35:13.395730 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:35:13.398539 (+ 2809us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:35:13.398554 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:35:13.398565 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:35:13.398587 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:35:13.400453 (+ 1866us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:35:13.401023 (+ 570us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:35:13.401029 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:35:13.401031 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:35:13.401147 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:35:13.401482 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:35:13.401487 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:35:13.401489 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:35:13.401876 (+ 387us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:35:13.402019 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0809 09:35:13.402381 (+ 362us) server_negotiation.cc:299] Negotiation successful
0809 09:35:13.402807 (+ 426us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":180,"server-negotiator.queue_time_us":188,"thread_start_us":115,"threads_started":1}
I20260809 09:35:13.406700 8403 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.125.189" port: 35185 } has no permanent_uuid. Determining permanent_uuid...
I20260809 09:35:13.407742 8405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:35:13.407323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.189:35185 (local address 127.0.0.1:33068)
0809 09:35:13.407509 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0809 09:35:13.407629 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.189:35185: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
W20260809 09:35:13.408032 8328 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.20.125.189:35185: connect: Connection refused (error 111)
I20260809 09:35:13.409030 8405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:35:13.408714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.189:35185 (local address 127.0.0.1:33082)
0809 09:35:13.408865 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0809 09:35:13.408947 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.189:35185: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260809 09:35:13.409226 8403 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.125.189:35185: Network error: Client connection negotiation failed: client connection to 127.20.125.189:35185: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.125.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.189.keytab.
Entry for principal kudu/127.20.125.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.189.keytab.
I20260809 09:35:13.417948 20982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
/tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786267941944493-20982-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786267941944493-20982-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786267941944493-20982-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786267941944493-20982-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.125.189:35185
--webserver_interface=127.20.125.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.189.keytab
--principal=kudu/127.20.125.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.125.148:45373
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.125.190:41293,127.20.125.189:35185,127.20.125.188:37597
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786267941944493-20982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786267941944493-20982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786267941944493-20982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 09:35:13.458545 8403 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.125.189" port: 35185 } attempt: 1
I20260809 09:35:13.459564 8405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:35:13.459185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.189:35185 (local address 127.0.0.1:33088)
0809 09:35:13.459381 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0809 09:35:13.459463 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.189:35185: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
I20260809 09:35:13.460449 8405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:35:13.460163 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.189:35185 (local address 127.0.0.1:33094)
0809 09:35:13.460317 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0809 09:35:13.460377 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.189:35185: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260809 09:35:13.460621 8403 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.125.189:35185: Network error: Client connection negotiation failed: client connection to 127.20.125.189:35185: connect: Connection refused (error 111)
I20260809 09:35:13.528831 8403 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.125.189" port: 35185 } attempt: 2
I20260809 09:35:13.529748 8405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:35:13.529412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.189:35185 (local address 127.0.0.1:33106)
0809 09:35:13.529599 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0809 09:35:13.529668 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.189:35185: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260809 09:35:13.530728 8405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:35:13.530438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.189:35185 (local address 127.0.0.1:33116)
0809 09:35:13.530607 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0809 09:35:13.530660 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.189:35185: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97}
W20260809 09:35:13.530900 8403 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.125.189:35185: Network error: Client connection negotiation failed: client connection to 127.20.125.189:35185: connect: Connection refused (error 111)
W20260809 09:35:13.534152 8415 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 09:35:13.534400 8415 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 09:35:13.534507 8415 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 09:35:13.538539 8415 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 09:35:13.538612 8415 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 09:35:13.538661 8415 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 09:35:13.538717 8415 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 09:35:13.538753 8415 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 09:35:13.548615 8415 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.125.148:45373
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786267941944493-20982-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786267941944493-20982-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.20.125.190:41293,127.20.125.189:35185,127.20.125.188:37597
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.189.keytab
--principal=kudu/127.20.125.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.125.189:35185
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786267941944493-20982-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.20.125.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8415
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786267941944493-20982-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 08:38:55 UTC on ccycloud.quasar-njjleu.root.comops.site
build id 4575935
I20260809 09:35:13.550027 8415 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 09:35:13.551400 8415 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 09:35:13.560129 8421 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:35:13.560129 8425 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:35:13.560129 8423 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:35:13.560547 8415 server_base.cc:1034] running on GCE node
I20260809 09:35:13.561124 8415 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 09:35:13.562216 8415 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 09:35:13.563411 8415 hybrid_clock.cc:648] HybridClock initialized: now 1786268113563378 us; error 52 us; skew 500 ppm
Aug 09 09:35:13 dist-test-slave-7kzw krb5kdc[8282](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268113, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 09:35:13.566386 8415 init.cc:377] Logged in from keytab as kudu/127.20.125.189@KRBTEST.COM (short username kudu)
I20260809 09:35:13.567368 8415 webserver.cc:460] Webserver started at http://127.20.125.189:37339/ using document root <none> and password file <none>
I20260809 09:35:13.567943 8415 fs_manager.cc:359] Metadata directory not provided
I20260809 09:35:13.568037 8415 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 09:35:13.568276 8415 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 09:35:13.570361 8415 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786267941944493-20982-0/minicluster-data/master-1/data/instance:
uuid: "39d58b63a475497da2d41ce04c04af9a"
format_stamp: "Formatted at 2026-08-09 09:35:13 on dist-test-slave-7kzw"
server_key: "cc87c398200ef29406e7fd5e7e134729"
server_key_iv: "c1b581b37880d4fe2c03dc6a15abcd79"
server_key_version: "encryptionkey@0"
I20260809 09:35:13.571008 8415 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786267941944493-20982-0/minicluster-data/master-1/wal/instance:
uuid: "39d58b63a475497da2d41ce04c04af9a"
format_stamp: "Formatted at 2026-08-09 09:35:13 on dist-test-slave-7kzw"
server_key: "cc87c398200ef29406e7fd5e7e134729"
server_key_iv: "c1b581b37880d4fe2c03dc6a15abcd79"
server_key_version: "encryptionkey@0"
I20260809 09:35:13.574751 8415 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260809 09:35:13.577272 8433 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 09:35:13.578682 8415 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 09:35:13.578835 8415 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786267941944493-20982-0/minicluster-data/master-1/data,/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786267941944493-20982-0/minicluster-data/master-1/wal
uuid: "39d58b63a475497da2d41ce04c04af9a"
format_stamp: "Formatted at 2026-08-09 09:35:13 on dist-test-slave-7kzw"
server_key: "cc87c398200ef29406e7fd5e7e134729"
server_key_iv: "c1b581b37880d4fe2c03dc6a15abcd79"
server_key_version: "encryptionkey@0"
I20260809 09:35:13.578954 8415 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786267941944493-20982-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786267941944493-20982-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786267941944493-20982-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 09:35:13.595865 8415 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 09:35:13.598907 8415 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 09:35:13.599116 8415 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 09:35:13.606127 8415 rpc_server.cc:307] RPC server started. Bound to: 127.20.125.189:35185
I20260809 09:35:13.606217 8509 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.125.189:35185 every 8 connection(s)
I20260809 09:35:13.607100 8415 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786267941944493-20982-0/minicluster-data/master-1/data/info.pb
I20260809 09:35:13.609925 8510 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 09:35:13.613776 8510 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.125.190" port: 41293 } has no permanent_uuid. Determining permanent_uuid...
I20260809 09:35:13.615883 20982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu as pid 8415
I20260809 09:35:13.616039 20982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786267941944493-20982-0/minicluster-data/master-1/wal/instance
I20260809 09:35:13.616319 20982 external_mini_cluster.cc:1428] Setting key e6ade9b20a24d8be2ccdd77454396d03
I20260809 09:35:13.622103 8403 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.125.189" port: 35185 } attempt: 3
Aug 09 09:35:13 dist-test-slave-7kzw krb5kdc[8282](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268113, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.189@KRBTEST.COM for kudu/127.20.125.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.20.125.188@KRBTEST.COM; defaulting to no policy
I20260809 09:35:13.630299 8409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:35:13.615727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35198 (local address 127.20.125.190:41293)
0809 09:35:13.615911 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0809 09:35:13.615916 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 09:35:13.616935 (+ 1019us) server_negotiation.cc:407] Connection header received
0809 09:35:13.618314 (+ 1379us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:35:13.618317 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:35:13.618406 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:35:13.618565 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:35:13.619712 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:35:13.620409 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:35:13.621553 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:35:13.621824 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:35:13.621941 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:35:13.624938 (+ 2997us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:35:13.624944 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:35:13.624947 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:35:13.624981 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:35:13.626657 (+ 1676us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:35:13.627672 (+ 1015us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:35:13.627677 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:35:13.627679 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:35:13.627774 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:35:13.628250 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:35:13.628255 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:35:13.628257 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:35:13.628469 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:35:13.628597 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0809 09:35:13.630003 (+ 1406us) server_negotiation.cc:299] Negotiation successful
0809 09:35:13.630146 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.125.188@KRBTEST.COM" created.
Aug 09 09:35:13 dist-test-slave-7kzw krb5kdc[8282](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268113, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.190@KRBTEST.COM for kudu/127.20.125.189@KRBTEST.COM
I20260809 09:35:13.630937 8513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:35:13.616198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:41293 (local address 127.0.0.1:35198)
0809 09:35:13.616747 (+ 549us) negotiation.cc:107] Waiting for socket to connect
0809 09:35:13.616811 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0809 09:35:13.618060 (+ 1249us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:35:13.618725 (+ 665us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:35:13.618732 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:35:13.619067 (+ 335us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:35:13.619540 (+ 473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:35:13.619553 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:35:13.620564 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:35:13.620566 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:35:13.621414 (+ 848us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:35:13.621424 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:35:13.621993 (+ 569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:35:13.621997 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:35:13.622198 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:35:13.623006 (+ 808us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 09:35:13.623030 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:35:13.624772 (+ 1742us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 09:35:13.626800 (+ 2028us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:35:13.626809 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:35:13.626825 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:35:13.627259 (+ 434us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:35:13.627893 (+ 634us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:35:13.627896 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:35:13.627898 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:35:13.628122 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:35:13.628593 (+ 471us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 09:35:13.628600 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 09:35:13.628909 (+ 309us) client_negotiation.cc:769] Sending connection context
0809 09:35:13.629121 (+ 212us) client_negotiation.cc:240] Negotiation successful
0809 09:35:13.629522 (+ 401us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":309,"thread_start_us":169,"threads_started":1}
I20260809 09:35:13.632751 8510 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.125.189" port: 35185 } has no permanent_uuid. Determining permanent_uuid...
I20260809 09:35:13.635526 8405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:35:13.622621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.189:35185 (local address 127.0.0.1:33130)
0809 09:35:13.622847 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0809 09:35:13.622863 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 09:35:13.622950 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:35:13.623656 (+ 706us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:35:13.623663 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:35:13.623900 (+ 237us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:35:13.624065 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:35:13.624076 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:35:13.625331 (+ 1255us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:35:13.625335 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:35:13.626316 (+ 981us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:35:13.626329 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:35:13.627105 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:35:13.627109 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:35:13.627223 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:35:13.627696 (+ 473us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 09:35:13.627711 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:35:13.631255 (+ 3544us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 09:35:13.633595 (+ 2340us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:35:13.633598 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:35:13.633600 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:35:13.633851 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:35:13.634233 (+ 382us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:35:13.634239 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:35:13.634244 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:35:13.634344 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:35:13.634956 (+ 612us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 09:35:13.634960 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 09:35:13.635065 (+ 105us) client_negotiation.cc:769] Sending connection context
0809 09:35:13.635215 (+ 150us) client_negotiation.cc:240] Negotiation successful
0809 09:35:13.635355 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138}
I20260809 09:35:13.635641 8516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:35:13.622714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33130 (local address 127.20.125.189:35185)
0809 09:35:13.623161 (+ 447us) server_negotiation.cc:206] Beginning negotiation
0809 09:35:13.623177 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 09:35:13.623201 (+ 24us) server_negotiation.cc:407] Connection header received
0809 09:35:13.623247 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:35:13.623255 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:35:13.623447 (+ 192us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:35:13.623620 (+ 173us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:35:13.624216 (+ 596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:35:13.625177 (+ 961us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:35:13.626486 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:35:13.626986 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:35:13.627103 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:35:13.631399 (+ 4296us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:35:13.631411 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:35:13.631426 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:35:13.631446 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:35:13.633476 (+ 2030us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:35:13.633977 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:35:13.633985 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:35:13.633987 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:35:13.634098 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:35:13.634449 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:35:13.634455 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:35:13.634458 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:35:13.634816 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:35:13.634980 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0809 09:35:13.635320 (+ 340us) server_negotiation.cc:299] Negotiation successful
0809 09:35:13.635457 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":348,"thread_start_us":138,"threads_started":1}
I20260809 09:35:13.637605 8403 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.125.188" port: 37597 } has no permanent_uuid. Determining permanent_uuid...
I20260809 09:35:13.638330 8405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:35:13.638085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.188:37597 (local address 127.0.0.1:42660)
0809 09:35:13.638207 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0809 09:35:13.638260 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.188:37597: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260809 09:35:13.639083 8405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:35:13.638833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.188:37597 (local address 127.0.0.1:42664)
0809 09:35:13.638977 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0809 09:35:13.639023 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.188:37597: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
W20260809 09:35:13.639230 8403 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.125.188:37597: Network error: Client connection negotiation failed: client connection to 127.20.125.188:37597: connect: Connection refused (error 111)
Aug 09 09:35:13 dist-test-slave-7kzw krb5kdc[8282](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268113, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.189@KRBTEST.COM for kudu/127.20.125.189@KRBTEST.COM
I20260809 09:35:13.643407 8513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:35:13.633271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.189:35185 (local address 127.0.0.1:33140)
0809 09:35:13.633360 (+ 89us) negotiation.cc:107] Waiting for socket to connect
0809 09:35:13.633375 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 09:35:13.633445 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:35:13.633962 (+ 517us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:35:13.633965 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:35:13.634101 (+ 136us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:35:13.634205 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:35:13.634212 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:35:13.635132 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:35:13.635137 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:35:13.636168 (+ 1031us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:35:13.636184 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:35:13.637312 (+ 1128us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:35:13.637320 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:35:13.637471 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:35:13.637974 (+ 503us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 09:35:13.637996 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:35:13.639772 (+ 1776us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 09:35:13.641653 (+ 1881us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:35:13.641657 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:35:13.641659 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:35:13.641857 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:35:13.642182 (+ 325us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:35:13.642187 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:35:13.642192 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:35:13.642280 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:35:13.642800 (+ 520us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 09:35:13.642803 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 09:35:13.642894 (+ 91us) client_negotiation.cc:769] Sending connection context
0809 09:35:13.642998 (+ 104us) client_negotiation.cc:240] Negotiation successful
0809 09:35:13.643219 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27}
I20260809 09:35:13.643462 8519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:35:13.633369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33140 (local address 127.20.125.189:35185)
0809 09:35:13.633731 (+ 362us) server_negotiation.cc:206] Beginning negotiation
0809 09:35:13.633742 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 09:35:13.633757 (+ 15us) server_negotiation.cc:407] Connection header received
0809 09:35:13.633791 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:35:13.633794 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:35:13.633836 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:35:13.634308 (+ 472us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:35:13.634386 (+ 78us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:35:13.634999 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:35:13.636311 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:35:13.637141 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:35:13.637300 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:35:13.639929 (+ 2629us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:35:13.639933 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:35:13.639936 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:35:13.639964 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:35:13.641523 (+ 1559us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:35:13.641957 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:35:13.641962 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:35:13.641964 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:35:13.642087 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:35:13.642377 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:35:13.642380 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:35:13.642382 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:35:13.642683 (+ 301us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:35:13.642800 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0809 09:35:13.643054 (+ 254us) server_negotiation.cc:299] Negotiation successful
0809 09:35:13.643286 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":124,"server-negotiator.queue_time_us":282,"thread_start_us":140,"threads_started":1}
I20260809 09:35:13.644347 8510 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.125.188" port: 37597 } has no permanent_uuid. Determining permanent_uuid...
I20260809 09:35:13.645000 8513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:35:13.644739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.188:37597 (local address 127.0.0.1:42672)
0809 09:35:13.644873 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0809 09:35:13.644942 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.188:37597: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
W20260809 09:35:13.645279 8435 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.20.125.188:37597: connect: Connection refused (error 111)
I20260809 09:35:13.646114 8513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:35:13.645944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.188:37597 (local address 127.0.0.1:42680)
0809 09:35:13.646024 (+ 80us) negotiation.cc:107] Waiting for socket to connect
0809 09:35:13.646067 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.188:37597: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
W20260809 09:35:13.646293 8510 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.125.188:37597: Network error: Client connection negotiation failed: client connection to 127.20.125.188:37597: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.125.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.188.keytab.
Entry for principal kudu/127.20.125.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.188.keytab.
I20260809 09:35:13.649696 20982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
/tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786267941944493-20982-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786267941944493-20982-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786267941944493-20982-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786267941944493-20982-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.125.188:37597
--webserver_interface=127.20.125.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.188.keytab
--principal=kudu/127.20.125.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.125.148:45373
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.125.190:41293,127.20.125.189:35185,127.20.125.188:37597
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786267941944493-20982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786267941944493-20982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786267941944493-20982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 09:35:13.670375 8403 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.125.188" port: 37597 } attempt: 1
I20260809 09:35:13.671607 8405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:35:13.671017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.188:37597 (local address 127.0.0.1:42694)
0809 09:35:13.671392 (+ 375us) negotiation.cc:107] Waiting for socket to connect
0809 09:35:13.671474 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.188:37597: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":227}
I20260809 09:35:13.672422 8405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:35:13.672198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.188:37597 (local address 127.0.0.1:42698)
0809 09:35:13.672307 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0809 09:35:13.672358 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.188:37597: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260809 09:35:13.672586 8403 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.125.188:37597: Network error: Client connection negotiation failed: client connection to 127.20.125.188:37597: connect: Connection refused (error 111)
I20260809 09:35:13.695626 8510 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.125.188" port: 37597 } attempt: 1
I20260809 09:35:13.696563 8513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:35:13.696254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.188:37597 (local address 127.0.0.1:42704)
0809 09:35:13.696417 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0809 09:35:13.696480 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.188:37597: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260809 09:35:13.697356 8513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:35:13.697125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.188:37597 (local address 127.0.0.1:42718)
0809 09:35:13.697244 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0809 09:35:13.697299 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.188:37597: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260809 09:35:13.697491 8510 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.125.188:37597: Network error: Client connection negotiation failed: client connection to 127.20.125.188:37597: connect: Connection refused (error 111)
I20260809 09:35:13.747762 8403 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.125.188" port: 37597 } attempt: 2
I20260809 09:35:13.748783 8405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:35:13.748365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.188:37597 (local address 127.0.0.1:42732)
0809 09:35:13.748590 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0809 09:35:13.748675 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.188:37597: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
I20260809 09:35:13.749704 8405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:35:13.749434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.188:37597 (local address 127.0.0.1:42742)
0809 09:35:13.749567 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0809 09:35:13.749626 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.188:37597: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260809 09:35:13.749897 8403 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.125.188:37597: Network error: Client connection negotiation failed: client connection to 127.20.125.188:37597: connect: Connection refused (error 111)
I20260809 09:35:13.765648 8510 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.125.188" port: 37597 } attempt: 2
I20260809 09:35:13.766644 8513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:35:13.766214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.188:37597 (local address 127.0.0.1:42748)
0809 09:35:13.766411 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0809 09:35:13.766517 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.188:37597: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260809 09:35:13.767408 8513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:35:13.767148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.188:37597 (local address 127.0.0.1:42752)
0809 09:35:13.767290 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0809 09:35:13.767346 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.188:37597: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260809 09:35:13.767573 8510 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.125.188:37597: Network error: Client connection negotiation failed: client connection to 127.20.125.188:37597: connect: Connection refused (error 111)
W20260809 09:35:13.769821 8521 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 09:35:13.770097 8521 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 09:35:13.770181 8521 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 09:35:13.774917 8521 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 09:35:13.774992 8521 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 09:35:13.775039 8521 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 09:35:13.775095 8521 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 09:35:13.775138 8521 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 09:35:13.780385 8521 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.125.148:45373
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786267941944493-20982-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786267941944493-20982-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.20.125.190:41293,127.20.125.189:35185,127.20.125.188:37597
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.188.keytab
--principal=kudu/127.20.125.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.125.188:37597
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786267941944493-20982-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.20.125.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8521
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786267941944493-20982-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 08:38:55 UTC on ccycloud.quasar-njjleu.root.comops.site
build id 4575935
I20260809 09:35:13.781438 8521 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 09:35:13.782552 8521 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 09:35:13.791255 8530 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:35:13.791330 8521 server_base.cc:1034] running on GCE node
W20260809 09:35:13.791286 8535 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:35:13.791545 8532 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:35:13.792161 8521 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 09:35:13.793393 8521 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 09:35:13.794574 8521 hybrid_clock.cc:648] HybridClock initialized: now 1786268113794530 us; error 58 us; skew 500 ppm
Aug 09 09:35:13 dist-test-slave-7kzw krb5kdc[8282](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268113, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 09:35:13.797502 8521 init.cc:377] Logged in from keytab as kudu/127.20.125.188@KRBTEST.COM (short username kudu)
I20260809 09:35:13.798511 8521 webserver.cc:460] Webserver started at http://127.20.125.188:46123/ using document root <none> and password file <none>
I20260809 09:35:13.799057 8521 fs_manager.cc:359] Metadata directory not provided
I20260809 09:35:13.799117 8521 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 09:35:13.799350 8521 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 09:35:13.801448 8521 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786267941944493-20982-0/minicluster-data/master-2/data/instance:
uuid: "479e2813570a46788046a6f240588987"
format_stamp: "Formatted at 2026-08-09 09:35:13 on dist-test-slave-7kzw"
server_key: "c232cb59b08f9a03f1f0b89abbb8af8a"
server_key_iv: "e25686c826f4d92c70d909755b65b60f"
server_key_version: "encryptionkey@0"
I20260809 09:35:13.801980 8521 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786267941944493-20982-0/minicluster-data/master-2/wal/instance:
uuid: "479e2813570a46788046a6f240588987"
format_stamp: "Formatted at 2026-08-09 09:35:13 on dist-test-slave-7kzw"
server_key: "c232cb59b08f9a03f1f0b89abbb8af8a"
server_key_iv: "e25686c826f4d92c70d909755b65b60f"
server_key_version: "encryptionkey@0"
I20260809 09:35:13.805703 8521 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260809 09:35:13.808143 8543 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 09:35:13.809497 8521 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260809 09:35:13.809612 8521 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786267941944493-20982-0/minicluster-data/master-2/data,/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786267941944493-20982-0/minicluster-data/master-2/wal
uuid: "479e2813570a46788046a6f240588987"
format_stamp: "Formatted at 2026-08-09 09:35:13 on dist-test-slave-7kzw"
server_key: "c232cb59b08f9a03f1f0b89abbb8af8a"
server_key_iv: "e25686c826f4d92c70d909755b65b60f"
server_key_version: "encryptionkey@0"
I20260809 09:35:13.809773 8521 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786267941944493-20982-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786267941944493-20982-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786267941944493-20982-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 09:35:13.827667 8521 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 09:35:13.830842 8521 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 09:35:13.831065 8521 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 09:35:13.838070 8521 rpc_server.cc:307] RPC server started. Bound to: 127.20.125.188:37597
I20260809 09:35:13.838377 8614 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.125.188:37597 every 8 connection(s)
I20260809 09:35:13.839093 8521 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786267941944493-20982-0/minicluster-data/master-2/data/info.pb
I20260809 09:35:13.842054 8615 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 09:35:13.846151 8615 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.125.190" port: 41293 } has no permanent_uuid. Determining permanent_uuid...
I20260809 09:35:13.847721 20982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu as pid 8521
I20260809 09:35:13.847858 20982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786267941944493-20982-0/minicluster-data/master-2/wal/instance
I20260809 09:35:13.848227 20982 external_mini_cluster.cc:1428] Setting key e818e1739aa5b029dbda92b0919285a0
I20260809 09:35:13.849095 8403 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.125.188" port: 37597 } attempt: 3
Aug 09 09:35:13 dist-test-slave-7kzw krb5kdc[8282](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268113, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.190@KRBTEST.COM for kudu/127.20.125.188@KRBTEST.COM
Aug 09 09:35:13 dist-test-slave-7kzw krb5kdc[8282](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268113, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.188@KRBTEST.COM for kudu/127.20.125.190@KRBTEST.COM
Aug 09 09:35:13 dist-test-slave-7kzw krb5kdc[8282](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268113, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.125.188@KRBTEST.COM
I20260809 09:35:13.858760 8510 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.125.188" port: 37597 } attempt: 3
I20260809 09:35:13.863018 8405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:35:13.849518 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.188:37597 (local address 127.0.0.1:42774)
0809 09:35:13.849714 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0809 09:35:13.849729 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 09:35:13.849828 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:35:13.851217 (+ 1389us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:35:13.851220 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:35:13.851470 (+ 250us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:35:13.851602 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:35:13.851612 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:35:13.853244 (+ 1632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:35:13.853250 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:35:13.854084 (+ 834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:35:13.854096 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:35:13.855057 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:35:13.855061 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:35:13.855164 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:35:13.855790 (+ 626us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 09:35:13.855812 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:35:13.857420 (+ 1608us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 09:35:13.860683 (+ 3263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:35:13.860689 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:35:13.860691 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:35:13.860998 (+ 307us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:35:13.861468 (+ 470us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:35:13.861474 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:35:13.861476 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:35:13.861605 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:35:13.862409 (+ 804us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 09:35:13.862412 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 09:35:13.862555 (+ 143us) client_negotiation.cc:769] Sending connection context
0809 09:35:13.862666 (+ 111us) client_negotiation.cc:240] Negotiation successful
0809 09:35:13.862808 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120}
I20260809 09:35:13.864220 8618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:35:13.848896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:41293 (local address 127.0.0.1:35206)
0809 09:35:13.849853 (+ 957us) negotiation.cc:107] Waiting for socket to connect
0809 09:35:13.849905 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0809 09:35:13.850730 (+ 825us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:35:13.851231 (+ 501us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:35:13.851237 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:35:13.851773 (+ 536us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:35:13.852132 (+ 359us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:35:13.852147 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:35:13.853312 (+ 1165us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:35:13.853316 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:35:13.854201 (+ 885us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:35:13.854216 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:35:13.854991 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:35:13.854996 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:35:13.855160 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:35:13.856002 (+ 842us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 09:35:13.856062 (+ 60us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:35:13.858240 (+ 2178us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 09:35:13.860744 (+ 2504us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:35:13.860756 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:35:13.860767 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:35:13.861165 (+ 398us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:35:13.861523 (+ 358us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:35:13.861528 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:35:13.861531 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:35:13.861730 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:35:13.862327 (+ 597us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 09:35:13.862337 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 09:35:13.862547 (+ 210us) client_negotiation.cc:769] Sending connection context
0809 09:35:13.862756 (+ 209us) client_negotiation.cc:240] Negotiation successful
0809 09:35:13.863091 (+ 335us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":385,"mutex_wait_us":160,"thread_start_us":146,"threads_started":1}
I20260809 09:35:13.864209 8620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:35:13.849896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42774 (local address 127.20.125.188:37597)
0809 09:35:13.850211 (+ 315us) server_negotiation.cc:206] Beginning negotiation
0809 09:35:13.850218 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 09:35:13.850229 (+ 11us) server_negotiation.cc:407] Connection header received
0809 09:35:13.850823 (+ 594us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:35:13.850831 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:35:13.851041 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:35:13.851750 (+ 709us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:35:13.851954 (+ 204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:35:13.853109 (+ 1155us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:35:13.854226 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:35:13.854911 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:35:13.855111 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:35:13.857673 (+ 2562us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:35:13.857684 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:35:13.857702 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:35:13.857731 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:35:13.860491 (+ 2760us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:35:13.861170 (+ 679us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:35:13.861176 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:35:13.861178 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:35:13.861349 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:35:13.861732 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:35:13.861737 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:35:13.861739 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:35:13.862287 (+ 548us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:35:13.863357 (+ 1070us) server_negotiation.cc:1035] Waiting for connection context
0809 09:35:13.863483 (+ 126us) server_negotiation.cc:299] Negotiation successful
0809 09:35:13.863639 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":59,"server-negotiator.queue_time_us":260,"spinlock_wait_cycles":5248,"thread_start_us":141,"threads_started":1}
I20260809 09:35:13.864692 8409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:35:13.848421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35206 (local address 127.20.125.190:41293)
0809 09:35:13.848603 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0809 09:35:13.848608 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 09:35:13.850016 (+ 1408us) server_negotiation.cc:407] Connection header received
0809 09:35:13.850989 (+ 973us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:35:13.850993 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:35:13.851041 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:35:13.851166 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:35:13.852282 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:35:13.853162 (+ 880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:35:13.854336 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:35:13.854715 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:35:13.854822 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:35:13.858450 (+ 3628us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:35:13.858453 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:35:13.858456 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:35:13.858509 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:35:13.860594 (+ 2085us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:35:13.861291 (+ 697us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:35:13.861295 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:35:13.861297 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:35:13.861399 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:35:13.861855 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:35:13.861859 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:35:13.861861 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:35:13.862167 (+ 306us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:35:13.862267 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0809 09:35:13.864334 (+ 2067us) server_negotiation.cc:299] Negotiation successful
0809 09:35:13.864482 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260809 09:35:13.865547 8619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:35:13.848967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42762 (local address 127.20.125.188:37597)
0809 09:35:13.849529 (+ 562us) server_negotiation.cc:206] Beginning negotiation
0809 09:35:13.849547 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0809 09:35:13.849590 (+ 43us) server_negotiation.cc:407] Connection header received
0809 09:35:13.850728 (+ 1138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:35:13.850773 (+ 45us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:35:13.851030 (+ 257us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:35:13.851246 (+ 216us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:35:13.852462 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:35:13.853311 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:35:13.854731 (+ 1420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:35:13.855116 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:35:13.855250 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:35:13.859678 (+ 4428us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:35:13.859685 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:35:13.859689 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:35:13.859716 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:35:13.862270 (+ 2554us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:35:13.862835 (+ 565us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:35:13.862838 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:35:13.862840 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:35:13.862921 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:35:13.863524 (+ 603us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:35:13.863527 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:35:13.863529 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:35:13.863739 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:35:13.863893 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0809 09:35:13.865261 (+ 1368us) server_negotiation.cc:299] Negotiation successful
0809 09:35:13.865372 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":7,"server-negotiator.queue_time_us":456,"spinlock_wait_cycles":99840,"thread_start_us":166,"threads_started":1}
Aug 09 09:35:13 dist-test-slave-7kzw krb5kdc[8282](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268113, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.189@KRBTEST.COM for kudu/127.20.125.188@KRBTEST.COM
I20260809 09:35:13.866860 8615 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.125.189" port: 35185 } has no permanent_uuid. Determining permanent_uuid...
I20260809 09:35:13.868973 8403 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 84d808263ac6480a9e1a73424b3da25e: Bootstrap starting.
Aug 09 09:35:13 dist-test-slave-7kzw krb5kdc[8282](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268113, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.188@KRBTEST.COM for kudu/127.20.125.189@KRBTEST.COM
I20260809 09:35:13.871759 8403 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 84d808263ac6480a9e1a73424b3da25e: Neither blocks nor log segments found. Creating new log.
I20260809 09:35:13.873123 8403 log.cc:824] T 00000000000000000000000000000000 P 84d808263ac6480a9e1a73424b3da25e: Log is configured to *not* fsync() on all Append() calls
I20260809 09:35:13.875909 8513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:35:13.859271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.188:37597 (local address 127.0.0.1:42782)
0809 09:35:13.859393 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0809 09:35:13.859410 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 09:35:13.859491 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:35:13.860550 (+ 1059us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:35:13.860554 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:35:13.860769 (+ 215us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:35:13.860910 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:35:13.860918 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:35:13.862419 (+ 1501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:35:13.862423 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:35:13.863335 (+ 912us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:35:13.863347 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:35:13.863991 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:35:13.863995 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:35:13.864085 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:35:13.864616 (+ 531us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 09:35:13.864633 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:35:13.866582 (+ 1949us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 09:35:13.873988 (+ 7406us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:35:13.873992 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:35:13.873994 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:35:13.874319 (+ 325us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:35:13.874741 (+ 422us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:35:13.874745 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:35:13.874747 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:35:13.874852 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:35:13.875366 (+ 514us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 09:35:13.875369 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 09:35:13.875476 (+ 107us) client_negotiation.cc:769] Sending connection context
0809 09:35:13.875587 (+ 111us) client_negotiation.cc:240] Negotiation successful
0809 09:35:13.875733 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260809 09:35:13.876116 8403 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 84d808263ac6480a9e1a73424b3da25e: No bootstrap required, opened a new log
I20260809 09:35:13.876444 8519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:35:13.867479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33150 (local address 127.20.125.189:35185)
0809 09:35:13.867602 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0809 09:35:13.867608 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 09:35:13.867624 (+ 16us) server_negotiation.cc:407] Connection header received
0809 09:35:13.867658 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:35:13.867661 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:35:13.867699 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:35:13.867790 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:35:13.868170 (+ 380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:35:13.868666 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:35:13.869517 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:35:13.869773 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:35:13.869878 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:35:13.872126 (+ 2248us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:35:13.872130 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:35:13.872133 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:35:13.872160 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:35:13.874074 (+ 1914us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:35:13.874658 (+ 584us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:35:13.874664 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:35:13.874669 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:35:13.874762 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:35:13.875079 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:35:13.875082 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:35:13.875084 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:35:13.875342 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:35:13.875411 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0809 09:35:13.875801 (+ 390us) server_negotiation.cc:299] Negotiation successful
0809 09:35:13.875939 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260809 09:35:13.876593 8621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:35:13.859573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42782 (local address 127.20.125.188:37597)
0809 09:35:13.859909 (+ 336us) server_negotiation.cc:206] Beginning negotiation
0809 09:35:13.860230 (+ 321us) server_negotiation.cc:399] Waiting for connection header
0809 09:35:13.860255 (+ 25us) server_negotiation.cc:407] Connection header received
0809 09:35:13.860302 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:35:13.860308 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:35:13.860366 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:35:13.861046 (+ 680us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:35:13.861177 (+ 131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:35:13.862280 (+ 1103us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:35:13.863499 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:35:13.863843 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:35:13.865260 (+ 1417us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:35:13.871997 (+ 6737us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:35:13.872004 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:35:13.872009 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:35:13.872042 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:35:13.873842 (+ 1800us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:35:13.874504 (+ 662us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:35:13.874509 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:35:13.874511 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:35:13.874612 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:35:13.874966 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:35:13.874969 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:35:13.874971 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:35:13.875206 (+ 235us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:35:13.875312 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0809 09:35:13.876281 (+ 969us) server_negotiation.cc:299] Negotiation successful
0809 09:35:13.876404 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":252,"thread_start_us":150,"threads_started":1}
I20260809 09:35:13.877264 8618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:35:13.867360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.189:35185 (local address 127.0.0.1:33150)
0809 09:35:13.867477 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0809 09:35:13.867491 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 09:35:13.867561 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:35:13.867806 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:35:13.867809 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:35:13.867971 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:35:13.868054 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:35:13.868059 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:35:13.868774 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:35:13.868779 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:35:13.869393 (+ 614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:35:13.869402 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:35:13.869891 (+ 489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:35:13.869895 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:35:13.870004 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:35:13.870488 (+ 484us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 09:35:13.870504 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:35:13.871968 (+ 1464us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 09:35:13.874205 (+ 2237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:35:13.874211 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:35:13.874214 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:35:13.874516 (+ 302us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:35:13.874880 (+ 364us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:35:13.874883 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:35:13.874884 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:35:13.874976 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:35:13.875491 (+ 515us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 09:35:13.875496 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 09:35:13.875611 (+ 115us) client_negotiation.cc:769] Sending connection context
0809 09:35:13.876943 (+ 1332us) client_negotiation.cc:240] Negotiation successful
0809 09:35:13.877109 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49}
I20260809 09:35:13.878418 8615 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.125.188" port: 37597 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 09:35:13 dist-test-slave-7kzw krb5kdc[8282](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268113, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.125.190@KRBTEST.COM
I20260809 09:35:13.879967 8510 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 39d58b63a475497da2d41ce04c04af9a: Bootstrap starting.
I20260809 09:35:13.881536 8403 raft_consensus.cc:348] T 00000000000000000000000000000000 P 84d808263ac6480a9e1a73424b3da25e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d808263ac6480a9e1a73424b3da25e" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 41293 } } peers { permanent_uuid: "39d58b63a475497da2d41ce04c04af9a" member_type: VOTER last_known_addr { host: "127.20.125.189" port: 35185 } } peers { permanent_uuid: "479e2813570a46788046a6f240588987" member_type: VOTER last_known_addr { host: "127.20.125.188" port: 37597 } }
I20260809 09:35:13.881803 8403 raft_consensus.cc:374] T 00000000000000000000000000000000 P 84d808263ac6480a9e1a73424b3da25e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 09:35:13.881865 8403 raft_consensus.cc:729] T 00000000000000000000000000000000 P 84d808263ac6480a9e1a73424b3da25e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84d808263ac6480a9e1a73424b3da25e, State: Initialized, Role: FOLLOWER
I20260809 09:35:13.882426 8403 consensus_queue.cc:260] T 00000000000000000000000000000000 P 84d808263ac6480a9e1a73424b3da25e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d808263ac6480a9e1a73424b3da25e" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 41293 } } peers { permanent_uuid: "39d58b63a475497da2d41ce04c04af9a" member_type: VOTER last_known_addr { host: "127.20.125.189" port: 35185 } } peers { permanent_uuid: "479e2813570a46788046a6f240588987" member_type: VOTER last_known_addr { host: "127.20.125.188" port: 37597 } }
I20260809 09:35:13.883487 8510 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 39d58b63a475497da2d41ce04c04af9a: Neither blocks nor log segments found. Creating new log.
I20260809 09:35:13.883728 8409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:35:13.872854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35208 (local address 127.20.125.190:41293)
0809 09:35:13.872996 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0809 09:35:13.873000 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 09:35:13.873377 (+ 377us) server_negotiation.cc:407] Connection header received
0809 09:35:13.873582 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:35:13.873585 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:35:13.873637 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:35:13.873691 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:35:13.874699 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:35:13.875429 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:35:13.876581 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:35:13.876922 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:35:13.877004 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:35:13.879956 (+ 2952us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:35:13.879960 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:35:13.879962 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:35:13.879988 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:35:13.881586 (+ 1598us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:35:13.882099 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:35:13.882105 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:35:13.882109 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:35:13.882200 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:35:13.882554 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:35:13.882559 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:35:13.882561 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:35:13.882803 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:35:13.882975 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0809 09:35:13.883419 (+ 444us) server_negotiation.cc:299] Negotiation successful
0809 09:35:13.883535 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 09:35:13.884009 8403 sys_catalog.cc:564] T 00000000000000000000000000000000 P 84d808263ac6480a9e1a73424b3da25e [sys.catalog]: configured and running, proceeding with master startup.
I20260809 09:35:13.884442 8510 log.cc:824] T 00000000000000000000000000000000 P 39d58b63a475497da2d41ce04c04af9a: Log is configured to *not* fsync() on all Append() calls
I20260809 09:35:13.886595 8627 sys_catalog.cc:455] T 00000000000000000000000000000000 P 84d808263ac6480a9e1a73424b3da25e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d808263ac6480a9e1a73424b3da25e" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 41293 } } peers { permanent_uuid: "39d58b63a475497da2d41ce04c04af9a" member_type: VOTER last_known_addr { host: "127.20.125.189" port: 35185 } } peers { permanent_uuid: "479e2813570a46788046a6f240588987" member_type: VOTER last_known_addr { host: "127.20.125.188" port: 37597 } } }
I20260809 09:35:13.886763 8627 sys_catalog.cc:458] T 00000000000000000000000000000000 P 84d808263ac6480a9e1a73424b3da25e [sys.catalog]: This master's current role is: FOLLOWER
I20260809 09:35:13.886986 8510 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 39d58b63a475497da2d41ce04c04af9a: No bootstrap required, opened a new log
I20260809 09:35:13.887527 8627 raft_consensus.cc:482] T 00000000000000000000000000000000 P 84d808263ac6480a9e1a73424b3da25e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
Aug 09 09:35:13 dist-test-slave-7kzw krb5kdc[8282](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268113, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.188@KRBTEST.COM for kudu/127.20.125.188@KRBTEST.COM
I20260809 09:35:13.887744 8627 raft_consensus.cc:504] T 00000000000000000000000000000000 P 84d808263ac6480a9e1a73424b3da25e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d808263ac6480a9e1a73424b3da25e" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 41293 } } peers { permanent_uuid: "39d58b63a475497da2d41ce04c04af9a" member_type: VOTER last_known_addr { host: "127.20.125.189" port: 35185 } } peers { permanent_uuid: "479e2813570a46788046a6f240588987" member_type: VOTER last_known_addr { host: "127.20.125.188" port: 37597 } }
I20260809 09:35:13.889808 8627 leader_election.cc:288] T 00000000000000000000000000000000 P 84d808263ac6480a9e1a73424b3da25e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 39d58b63a475497da2d41ce04c04af9a (127.20.125.189:35185), 479e2813570a46788046a6f240588987 (127.20.125.188:37597)
I20260809 09:35:13.891980 8510 raft_consensus.cc:348] T 00000000000000000000000000000000 P 39d58b63a475497da2d41ce04c04af9a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d808263ac6480a9e1a73424b3da25e" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 41293 } } peers { permanent_uuid: "39d58b63a475497da2d41ce04c04af9a" member_type: VOTER last_known_addr { host: "127.20.125.189" port: 35185 } } peers { permanent_uuid: "479e2813570a46788046a6f240588987" member_type: VOTER last_known_addr { host: "127.20.125.188" port: 37597 } }
I20260809 09:35:13.892257 8510 raft_consensus.cc:374] T 00000000000000000000000000000000 P 39d58b63a475497da2d41ce04c04af9a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 09:35:13.892328 8510 raft_consensus.cc:729] T 00000000000000000000000000000000 P 39d58b63a475497da2d41ce04c04af9a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39d58b63a475497da2d41ce04c04af9a, State: Initialized, Role: FOLLOWER
Aug 09 09:35:13 dist-test-slave-7kzw krb5kdc[8282](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268113, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.125.189@KRBTEST.COM
I20260809 09:35:13.892976 8621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:35:13.879159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42788 (local address 127.20.125.188:37597)
0809 09:35:13.882234 (+ 3075us) server_negotiation.cc:206] Beginning negotiation
0809 09:35:13.882239 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 09:35:13.882255 (+ 16us) server_negotiation.cc:407] Connection header received
0809 09:35:13.882291 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:35:13.882294 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:35:13.882360 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:35:13.882451 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:35:13.883024 (+ 573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:35:13.883757 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:35:13.885095 (+ 1338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:35:13.885411 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:35:13.885495 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:35:13.888217 (+ 2722us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:35:13.888220 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:35:13.888223 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:35:13.888246 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:35:13.890112 (+ 1866us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:35:13.890698 (+ 586us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:35:13.890702 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:35:13.890705 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:35:13.890793 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:35:13.891408 (+ 615us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:35:13.891412 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:35:13.891414 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:35:13.891652 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:35:13.891747 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0809 09:35:13.892666 (+ 919us) server_negotiation.cc:299] Negotiation successful
0809 09:35:13.892802 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2998}
I20260809 09:35:13.892900 8510 consensus_queue.cc:260] T 00000000000000000000000000000000 P 39d58b63a475497da2d41ce04c04af9a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d808263ac6480a9e1a73424b3da25e" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 41293 } } peers { permanent_uuid: "39d58b63a475497da2d41ce04c04af9a" member_type: VOTER last_known_addr { host: "127.20.125.189" port: 35185 } } peers { permanent_uuid: "479e2813570a46788046a6f240588987" member_type: VOTER last_known_addr { host: "127.20.125.188" port: 37597 } }
I20260809 09:35:13.894104 8618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:35:13.878947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.188:37597 (local address 127.0.0.1:42788)
0809 09:35:13.879265 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0809 09:35:13.879278 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 09:35:13.879350 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:35:13.882503 (+ 3153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:35:13.882507 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:35:13.882806 (+ 299us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:35:13.882907 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:35:13.882914 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:35:13.884181 (+ 1267us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:35:13.884184 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:35:13.884973 (+ 789us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:35:13.884984 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:35:13.885710 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:35:13.885713 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:35:13.885813 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:35:13.886297 (+ 484us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 09:35:13.886311 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:35:13.888080 (+ 1769us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 09:35:13.890249 (+ 2169us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:35:13.890252 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:35:13.890254 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:35:13.890567 (+ 313us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:35:13.890942 (+ 375us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:35:13.890946 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:35:13.890948 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:35:13.891209 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:35:13.891866 (+ 657us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 09:35:13.891870 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 09:35:13.892079 (+ 209us) client_negotiation.cc:769] Sending connection context
0809 09:35:13.893443 (+ 1364us) client_negotiation.cc:240] Negotiation successful
0809 09:35:13.893774 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":247}
I20260809 09:35:13.894439 8510 sys_catalog.cc:564] T 00000000000000000000000000000000 P 39d58b63a475497da2d41ce04c04af9a [sys.catalog]: configured and running, proceeding with master startup.
I20260809 09:35:13.894829 8466 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "84d808263ac6480a9e1a73424b3da25e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39d58b63a475497da2d41ce04c04af9a" is_pre_election: true
I20260809 09:35:13.895138 8466 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 39d58b63a475497da2d41ce04c04af9a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d808263ac6480a9e1a73424b3da25e in term 0.
I20260809 09:35:13.897166 8519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:35:13.885476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33162 (local address 127.20.125.189:35185)
0809 09:35:13.885592 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0809 09:35:13.885596 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 09:35:13.885784 (+ 188us) server_negotiation.cc:407] Connection header received
0809 09:35:13.886248 (+ 464us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:35:13.886253 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:35:13.886312 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:35:13.886372 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:35:13.887313 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:35:13.888070 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:35:13.889775 (+ 1705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:35:13.890152 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:35:13.890233 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:35:13.893217 (+ 2984us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:35:13.893220 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:35:13.893223 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:35:13.893245 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:35:13.894978 (+ 1733us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:35:13.895497 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:35:13.895500 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:35:13.895502 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:35:13.895618 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:35:13.895938 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:35:13.895941 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:35:13.895943 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:35:13.896167 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:35:13.896376 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0809 09:35:13.896863 (+ 487us) server_negotiation.cc:299] Negotiation successful
0809 09:35:13.897007 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260809 09:35:13.897665 8615 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 479e2813570a46788046a6f240588987: Bootstrap starting.
I20260809 09:35:13.898202 8328 leader_election.cc:302] T 00000000000000000000000000000000 P 84d808263ac6480a9e1a73424b3da25e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39d58b63a475497da2d41ce04c04af9a, 84d808263ac6480a9e1a73424b3da25e; no voters:
I20260809 09:35:13.898576 8627 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 84d808263ac6480a9e1a73424b3da25e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 09:35:13.898705 8627 raft_consensus.cc:482] T 00000000000000000000000000000000 P 84d808263ac6480a9e1a73424b3da25e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 09:35:13.898767 8627 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 84d808263ac6480a9e1a73424b3da25e [term 0 FOLLOWER]: Advancing to term 1
I20260809 09:35:13.893733 8301 master_runner.cc:186] Error getting master registration for 127.20.125.189:35185: OK, instance_id { permanent_uuid: "39d58b63a475497da2d41ce04c04af9a" instance_seqno: 1786268113604128 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 09:35:13.899923 8301 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 09:35:13.900193 8627 raft_consensus.cc:504] T 00000000000000000000000000000000 P 84d808263ac6480a9e1a73424b3da25e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d808263ac6480a9e1a73424b3da25e" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 41293 } } peers { permanent_uuid: "39d58b63a475497da2d41ce04c04af9a" member_type: VOTER last_known_addr { host: "127.20.125.189" port: 35185 } } peers { permanent_uuid: "479e2813570a46788046a6f240588987" member_type: VOTER last_known_addr { host: "127.20.125.188" port: 37597 } }
I20260809 09:35:13.900614 20982 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 09:35:13.900589 8627 leader_election.cc:288] T 00000000000000000000000000000000 P 84d808263ac6480a9e1a73424b3da25e [CANDIDATE]: Term 1 election: Requested vote from peers 39d58b63a475497da2d41ce04c04af9a (127.20.125.189:35185), 479e2813570a46788046a6f240588987 (127.20.125.188:37597)
I20260809 09:35:13.900794 8615 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 479e2813570a46788046a6f240588987: Neither blocks nor log segments found. Creating new log.
I20260809 09:35:13.900914 8466 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "84d808263ac6480a9e1a73424b3da25e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39d58b63a475497da2d41ce04c04af9a"
I20260809 09:35:13.901026 8466 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 39d58b63a475497da2d41ce04c04af9a [term 0 FOLLOWER]: Advancing to term 1
I20260809 09:35:13.901968 8615 log.cc:824] T 00000000000000000000000000000000 P 479e2813570a46788046a6f240588987: Log is configured to *not* fsync() on all Append() calls
I20260809 09:35:13.902400 8466 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 39d58b63a475497da2d41ce04c04af9a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d808263ac6480a9e1a73424b3da25e in term 1.
I20260809 09:35:13.902755 8328 leader_election.cc:302] T 00000000000000000000000000000000 P 84d808263ac6480a9e1a73424b3da25e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39d58b63a475497da2d41ce04c04af9a, 84d808263ac6480a9e1a73424b3da25e; no voters:
I20260809 09:35:13.902935 8627 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 84d808263ac6480a9e1a73424b3da25e [term 1 FOLLOWER]: Leader election won for term 1
I20260809 09:35:13.903182 8627 raft_consensus.cc:686] T 00000000000000000000000000000000 P 84d808263ac6480a9e1a73424b3da25e [term 1 LEADER]: Becoming Leader. State: Replica: 84d808263ac6480a9e1a73424b3da25e, State: Running, Role: LEADER
I20260809 09:35:13.903541 8627 consensus_queue.cc:237] T 00000000000000000000000000000000 P 84d808263ac6480a9e1a73424b3da25e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d808263ac6480a9e1a73424b3da25e" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 41293 } } peers { permanent_uuid: "39d58b63a475497da2d41ce04c04af9a" member_type: VOTER last_known_addr { host: "127.20.125.189" port: 35185 } } peers { permanent_uuid: "479e2813570a46788046a6f240588987" member_type: VOTER last_known_addr { host: "127.20.125.188" port: 37597 } }
I20260809 09:35:13.905220 8615 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 479e2813570a46788046a6f240588987: No bootstrap required, opened a new log
I20260809 09:35:13.905347 8627 sys_catalog.cc:455] T 00000000000000000000000000000000 P 84d808263ac6480a9e1a73424b3da25e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 84d808263ac6480a9e1a73424b3da25e. Latest consensus state: current_term: 1 leader_uuid: "84d808263ac6480a9e1a73424b3da25e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d808263ac6480a9e1a73424b3da25e" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 41293 } } peers { permanent_uuid: "39d58b63a475497da2d41ce04c04af9a" member_type: VOTER last_known_addr { host: "127.20.125.189" port: 35185 } } peers { permanent_uuid: "479e2813570a46788046a6f240588987" member_type: VOTER last_known_addr { host: "127.20.125.188" port: 37597 } } }
I20260809 09:35:13.905473 8627 sys_catalog.cc:458] T 00000000000000000000000000000000 P 84d808263ac6480a9e1a73424b3da25e [sys.catalog]: This master's current role is: LEADER
I20260809 09:35:13.906528 8647 sys_catalog.cc:455] T 00000000000000000000000000000000 P 39d58b63a475497da2d41ce04c04af9a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d808263ac6480a9e1a73424b3da25e" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 41293 } } peers { permanent_uuid: "39d58b63a475497da2d41ce04c04af9a" member_type: VOTER last_known_addr { host: "127.20.125.189" port: 35185 } } peers { permanent_uuid: "479e2813570a46788046a6f240588987" member_type: VOTER last_known_addr { host: "127.20.125.188" port: 37597 } } }
I20260809 09:35:13.906682 8647 sys_catalog.cc:458] T 00000000000000000000000000000000 P 39d58b63a475497da2d41ce04c04af9a [sys.catalog]: This master's current role is: FOLLOWER
I20260809 09:35:13.910068 8615 raft_consensus.cc:348] T 00000000000000000000000000000000 P 479e2813570a46788046a6f240588987 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d808263ac6480a9e1a73424b3da25e" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 41293 } } peers { permanent_uuid: "39d58b63a475497da2d41ce04c04af9a" member_type: VOTER last_known_addr { host: "127.20.125.189" port: 35185 } } peers { permanent_uuid: "479e2813570a46788046a6f240588987" member_type: VOTER last_known_addr { host: "127.20.125.188" port: 37597 } }
I20260809 09:35:13.910328 8615 raft_consensus.cc:374] T 00000000000000000000000000000000 P 479e2813570a46788046a6f240588987 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 09:35:13.910385 8615 raft_consensus.cc:729] T 00000000000000000000000000000000 P 479e2813570a46788046a6f240588987 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 479e2813570a46788046a6f240588987, State: Initialized, Role: FOLLOWER
I20260809 09:35:13.910290 8415 master_runner.cc:186] Error getting master registration for 127.20.125.188:37597: OK, instance_id { permanent_uuid: "479e2813570a46788046a6f240588987" instance_seqno: 1786268113836171 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 09:35:13.913796 8415 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260809 09:35:13.913630 8676 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 39d58b63a475497da2d41ce04c04af9a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 09:35:13.914186 8676 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 09:35:13.915623 8615 consensus_queue.cc:260] T 00000000000000000000000000000000 P 479e2813570a46788046a6f240588987 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d808263ac6480a9e1a73424b3da25e" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 41293 } } peers { permanent_uuid: "39d58b63a475497da2d41ce04c04af9a" member_type: VOTER last_known_addr { host: "127.20.125.189" port: 35185 } } peers { permanent_uuid: "479e2813570a46788046a6f240588987" member_type: VOTER last_known_addr { host: "127.20.125.188" port: 37597 } }
W20260809 09:35:13.916307 8645 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 84d808263ac6480a9e1a73424b3da25e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 09:35:13.916386 8645 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 09:35:13.916446 8678 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 09:35:13.917035 8615 sys_catalog.cc:564] T 00000000000000000000000000000000 P 479e2813570a46788046a6f240588987 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 09:35:13.917089 8578 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "84d808263ac6480a9e1a73424b3da25e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "479e2813570a46788046a6f240588987"
I20260809 09:35:13.917310 8678 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 09:35:13.917352 8578 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 479e2813570a46788046a6f240588987 [term 0 FOLLOWER]: Advancing to term 1
I20260809 09:35:13.918882 8578 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 479e2813570a46788046a6f240588987 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d808263ac6480a9e1a73424b3da25e in term 1.
I20260809 09:35:13.919193 8579 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "84d808263ac6480a9e1a73424b3da25e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "479e2813570a46788046a6f240588987" is_pre_election: true
I20260809 09:35:13.919342 8579 raft_consensus.cc:2362] T 00000000000000000000000000000000 P 479e2813570a46788046a6f240588987 [term 1 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 84d808263ac6480a9e1a73424b3da25e in term 1. Re-sending same reply.
I20260809 09:35:13.923216 8519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:35:13.908891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33174 (local address 127.20.125.189:35185)
0809 09:35:13.909002 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0809 09:35:13.909007 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 09:35:13.910893 (+ 1886us) server_negotiation.cc:407] Connection header received
0809 09:35:13.911065 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:35:13.911068 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:35:13.911111 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:35:13.911159 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:35:13.912100 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:35:13.912881 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:35:13.914189 (+ 1308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:35:13.914514 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:35:13.914614 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:35:13.919466 (+ 4852us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:35:13.919473 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:35:13.919479 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:35:13.919508 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:35:13.921222 (+ 1714us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:35:13.921734 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:35:13.921738 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:35:13.921740 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:35:13.921826 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:35:13.922147 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:35:13.922154 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:35:13.922157 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:35:13.922385 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:35:13.922524 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0809 09:35:13.922922 (+ 398us) server_negotiation.cc:299] Negotiation successful
0809 09:35:13.923058 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260809 09:35:13.923632 8579 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 479e2813570a46788046a6f240588987 [term 1 FOLLOWER]: Refusing update from remote peer 84d808263ac6480a9e1a73424b3da25e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 09:35:13.924294 8660 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 84d808263ac6480a9e1a73424b3da25e [LEADER]: Connected to new peer: Peer: permanent_uuid: "479e2813570a46788046a6f240588987" member_type: VOTER last_known_addr { host: "127.20.125.188" port: 37597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 09:35:13.929044 8466 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 39d58b63a475497da2d41ce04c04af9a [term 1 FOLLOWER]: Refusing update from remote peer 84d808263ac6480a9e1a73424b3da25e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 09:35:13.929658 8660 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 84d808263ac6480a9e1a73424b3da25e [LEADER]: Connected to new peer: Peer: permanent_uuid: "39d58b63a475497da2d41ce04c04af9a" member_type: VOTER last_known_addr { host: "127.20.125.189" port: 35185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 09:35:13.930102 8409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:35:13.903503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35214 (local address 127.20.125.190:41293)
0809 09:35:13.903637 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0809 09:35:13.903642 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 09:35:13.910261 (+ 6619us) server_negotiation.cc:407] Connection header received
0809 09:35:13.910514 (+ 253us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:35:13.910518 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:35:13.910573 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:35:13.910657 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:35:13.914991 (+ 4334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:35:13.916209 (+ 1218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:35:13.921222 (+ 5013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:35:13.921575 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:35:13.921665 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:35:13.923672 (+ 2007us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:35:13.923677 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:35:13.923680 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:35:13.923707 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:35:13.925635 (+ 1928us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:35:13.928550 (+ 2915us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:35:13.928554 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:35:13.928556 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:35:13.928658 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:35:13.928999 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:35:13.929002 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:35:13.929003 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:35:13.929232 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:35:13.929360 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0809 09:35:13.929802 (+ 442us) server_negotiation.cc:299] Negotiation successful
0809 09:35:13.929927 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
W20260809 09:35:13.932747 8705 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 479e2813570a46788046a6f240588987: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 09:35:13.932834 8705 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 09:35:13.934015 8647 sys_catalog.cc:455] T 00000000000000000000000000000000 P 39d58b63a475497da2d41ce04c04af9a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 84d808263ac6480a9e1a73424b3da25e. Latest consensus state: current_term: 1 leader_uuid: "84d808263ac6480a9e1a73424b3da25e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d808263ac6480a9e1a73424b3da25e" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 41293 } } peers { permanent_uuid: "39d58b63a475497da2d41ce04c04af9a" member_type: VOTER last_known_addr { host: "127.20.125.189" port: 35185 } } peers { permanent_uuid: "479e2813570a46788046a6f240588987" member_type: VOTER last_known_addr { host: "127.20.125.188" port: 37597 } } }
I20260809 09:35:13.934162 8647 sys_catalog.cc:458] T 00000000000000000000000000000000 P 39d58b63a475497da2d41ce04c04af9a [sys.catalog]: This master's current role is: FOLLOWER
I20260809 09:35:13.934451 8684 sys_catalog.cc:455] T 00000000000000000000000000000000 P 479e2813570a46788046a6f240588987 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "84d808263ac6480a9e1a73424b3da25e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d808263ac6480a9e1a73424b3da25e" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 41293 } } peers { permanent_uuid: "39d58b63a475497da2d41ce04c04af9a" member_type: VOTER last_known_addr { host: "127.20.125.189" port: 35185 } } peers { permanent_uuid: "479e2813570a46788046a6f240588987" member_type: VOTER last_known_addr { host: "127.20.125.188" port: 37597 } } }
I20260809 09:35:13.934610 8684 sys_catalog.cc:458] T 00000000000000000000000000000000 P 479e2813570a46788046a6f240588987 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 09:35:13.934729 8684 sys_catalog.cc:455] T 00000000000000000000000000000000 P 479e2813570a46788046a6f240588987 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 84d808263ac6480a9e1a73424b3da25e. Latest consensus state: current_term: 1 leader_uuid: "84d808263ac6480a9e1a73424b3da25e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d808263ac6480a9e1a73424b3da25e" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 41293 } } peers { permanent_uuid: "39d58b63a475497da2d41ce04c04af9a" member_type: VOTER last_known_addr { host: "127.20.125.189" port: 35185 } } peers { permanent_uuid: "479e2813570a46788046a6f240588987" member_type: VOTER last_known_addr { host: "127.20.125.188" port: 37597 } } }
I20260809 09:35:13.934818 8684 sys_catalog.cc:458] T 00000000000000000000000000000000 P 479e2813570a46788046a6f240588987 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 09:35:13.937206 8660 sys_catalog.cc:455] T 00000000000000000000000000000000 P 84d808263ac6480a9e1a73424b3da25e [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "84d808263ac6480a9e1a73424b3da25e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d808263ac6480a9e1a73424b3da25e" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 41293 } } peers { permanent_uuid: "39d58b63a475497da2d41ce04c04af9a" member_type: VOTER last_known_addr { host: "127.20.125.189" port: 35185 } } peers { permanent_uuid: "479e2813570a46788046a6f240588987" member_type: VOTER last_known_addr { host: "127.20.125.188" port: 37597 } } }
I20260809 09:35:13.937342 8660 sys_catalog.cc:458] T 00000000000000000000000000000000 P 84d808263ac6480a9e1a73424b3da25e [sys.catalog]: This master's current role is: LEADER
I20260809 09:35:13.938046 8627 sys_catalog.cc:455] T 00000000000000000000000000000000 P 84d808263ac6480a9e1a73424b3da25e [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "84d808263ac6480a9e1a73424b3da25e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d808263ac6480a9e1a73424b3da25e" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 41293 } } peers { permanent_uuid: "39d58b63a475497da2d41ce04c04af9a" member_type: VOTER last_known_addr { host: "127.20.125.189" port: 35185 } } peers { permanent_uuid: "479e2813570a46788046a6f240588987" member_type: VOTER last_known_addr { host: "127.20.125.188" port: 37597 } } }
I20260809 09:35:13.938153 8627 sys_catalog.cc:458] T 00000000000000000000000000000000 P 84d808263ac6480a9e1a73424b3da25e [sys.catalog]: This master's current role is: LEADER
I20260809 09:35:13.938521 8647 sys_catalog.cc:455] T 00000000000000000000000000000000 P 39d58b63a475497da2d41ce04c04af9a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "84d808263ac6480a9e1a73424b3da25e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d808263ac6480a9e1a73424b3da25e" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 41293 } } peers { permanent_uuid: "39d58b63a475497da2d41ce04c04af9a" member_type: VOTER last_known_addr { host: "127.20.125.189" port: 35185 } } peers { permanent_uuid: "479e2813570a46788046a6f240588987" member_type: VOTER last_known_addr { host: "127.20.125.188" port: 37597 } } }
I20260809 09:35:13.938645 8647 sys_catalog.cc:458] T 00000000000000000000000000000000 P 39d58b63a475497da2d41ce04c04af9a [sys.catalog]: This master's current role is: FOLLOWER
I20260809 09:35:13.939419 8621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:35:13.909640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42792 (local address 127.20.125.188:37597)
0809 09:35:13.909752 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0809 09:35:13.909759 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 09:35:13.925082 (+ 15323us) server_negotiation.cc:407] Connection header received
0809 09:35:13.925261 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:35:13.925264 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:35:13.925313 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:35:13.925363 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:35:13.926539 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:35:13.927321 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:35:13.931151 (+ 3830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:35:13.931469 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:35:13.931563 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:35:13.933435 (+ 1872us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:35:13.933439 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:35:13.933441 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:35:13.933466 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:35:13.936929 (+ 3463us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:35:13.937492 (+ 563us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:35:13.937495 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:35:13.937497 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:35:13.937600 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:35:13.937951 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:35:13.937954 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:35:13.937956 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:35:13.938357 (+ 401us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:35:13.938553 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0809 09:35:13.939087 (+ 534us) server_negotiation.cc:299] Negotiation successful
0809 09:35:13.939246 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260809 09:35:13.939590 8702 mvcc.cc:205] Tried to move back new op lower bound from 7316554194623004672 to 7316554194552086528. Current Snapshot: MvccSnapshot[applied={T|T < 7316554194623004672}]
I20260809 09:35:13.940243 8693 sys_catalog.cc:455] T 00000000000000000000000000000000 P 479e2813570a46788046a6f240588987 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "84d808263ac6480a9e1a73424b3da25e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d808263ac6480a9e1a73424b3da25e" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 41293 } } peers { permanent_uuid: "39d58b63a475497da2d41ce04c04af9a" member_type: VOTER last_known_addr { host: "127.20.125.189" port: 35185 } } peers { permanent_uuid: "479e2813570a46788046a6f240588987" member_type: VOTER last_known_addr { host: "127.20.125.188" port: 37597 } } }
I20260809 09:35:13.940384 8693 sys_catalog.cc:458] T 00000000000000000000000000000000 P 479e2813570a46788046a6f240588987 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 09:35:13.944597 8710 mvcc.cc:205] Tried to move back new op lower bound from 7316554194623004672 to 7316554194552086528. Current Snapshot: MvccSnapshot[applied={T|T < 7316554194623004672}]
I20260809 09:35:13.946578 8678 catalog_manager.cc:1347] Generated new cluster ID: d34439ea7a1041aeaa51161ca3bce647
I20260809 09:35:13.946664 8678 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 09:35:13.994580 8678 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 09:35:13.995973 8678 catalog_manager.cc:1504] Loading token signing keys...
I20260809 09:35:14.019212 8678 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 84d808263ac6480a9e1a73424b3da25e: Generated new TSK 0
I20260809 09:35:14.020262 8678 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 09:35:14.916029 8676 catalog_manager.cc:1259] Loaded cluster ID: d34439ea7a1041aeaa51161ca3bce647
I20260809 09:35:14.916150 8676 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 39d58b63a475497da2d41ce04c04af9a: loading cluster ID for follower catalog manager: success
I20260809 09:35:14.918128 8676 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 39d58b63a475497da2d41ce04c04af9a: acquiring CA information for follower catalog manager: success
I20260809 09:35:14.919432 8676 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 39d58b63a475497da2d41ce04c04af9a: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 09:35:14.933987 8705 catalog_manager.cc:1259] Loaded cluster ID: d34439ea7a1041aeaa51161ca3bce647
I20260809 09:35:14.934091 8705 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 479e2813570a46788046a6f240588987: loading cluster ID for follower catalog manager: success
I20260809 09:35:14.935765 8705 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 479e2813570a46788046a6f240588987: acquiring CA information for follower catalog manager: success
I20260809 09:35:14.936941 8705 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 479e2813570a46788046a6f240588987: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 09:35:15.089282 20982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu with pid 8301
I20260809 09:35:15.102617 20982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu with pid 8415
I20260809 09:35:15.113893 20982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu with pid 8521
2026-08-09T09:35:15Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (3336 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786267941944493-20982-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 09:35:15 dist-test-slave-7kzw krb5kdc[8729](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 09:35:15 dist-test-slave-7kzw krb5kdc[8729](info): set up 2 sockets
Aug 09 09:35:15 dist-test-slave-7kzw krb5kdc[8729](info): commencing operation
krb5kdc: starting...
W20260809 09:35:16.377305 20982 mini_kdc.cc:121] Time spent starting KDC: real 1.232s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786267941944493-20982-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786267941944493-20982-0/krb5kdc/test-user.keytab.
Aug 09 09:35:16 dist-test-slave-7kzw krb5kdc[8729](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268116, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T09:35:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T09:35:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.125.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.125.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.125.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.190.keytab.
Entry for principal kudu/127.20.125.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.190.keytab.
I20260809 09:35:16.518203 20982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
/tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786267941944493-20982-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786267941944493-20982-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786267941944493-20982-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786267941944493-20982-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.125.190:33057
--webserver_interface=127.20.125.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.190.keytab
--principal=kudu/127.20.125.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.125.148:38523
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.125.190:33057
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786267941944493-20982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786267941944493-20982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786267941944493-20982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 09:35:16.635414 8745 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 09:35:16.635701 8745 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 09:35:16.635741 8745 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 09:35:16.639750 8745 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 09:35:16.639814 8745 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 09:35:16.639834 8745 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 09:35:16.639851 8745 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 09:35:16.639866 8745 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 09:35:16.645274 8745 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.125.148:38523
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786267941944493-20982-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786267941944493-20982-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.125.190:33057
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.190.keytab
--principal=kudu/127.20.125.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.125.190:33057
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786267941944493-20982-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.125.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8745
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786267941944493-20982-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 08:38:55 UTC on ccycloud.quasar-njjleu.root.comops.site
build id 4575935
I20260809 09:35:16.646417 8745 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 09:35:16.647557 8745 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 09:35:16.656812 8759 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:35:16.656852 8757 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:35:16.657016 8745 server_base.cc:1034] running on GCE node
W20260809 09:35:16.656852 8761 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:35:16.657744 8745 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 09:35:16.658983 8745 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 09:35:16.660194 8745 hybrid_clock.cc:648] HybridClock initialized: now 1786268116660144 us; error 60 us; skew 500 ppm
Aug 09 09:35:16 dist-test-slave-7kzw krb5kdc[8729](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268116, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 09:35:16.663300 8745 init.cc:377] Logged in from keytab as kudu/127.20.125.190@KRBTEST.COM (short username kudu)
I20260809 09:35:16.664181 8745 webserver.cc:460] Webserver started at http://127.20.125.190:46347/ using document root <none> and password file <none>
I20260809 09:35:16.664763 8745 fs_manager.cc:359] Metadata directory not provided
I20260809 09:35:16.664829 8745 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 09:35:16.665058 8745 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 09:35:16.667233 8745 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786267941944493-20982-0/minicluster-data/master-0/data/instance:
uuid: "72f5995bd52b4bae956af3750631ef08"
format_stamp: "Formatted at 2026-08-09 09:35:16 on dist-test-slave-7kzw"
server_key: "55067c0f8f10256574c84eefa161ed53"
server_key_iv: "47ae416abbdb75233353503d512d9ef4"
server_key_version: "encryptionkey@0"
I20260809 09:35:16.667841 8745 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786267941944493-20982-0/minicluster-data/master-0/wal/instance:
uuid: "72f5995bd52b4bae956af3750631ef08"
format_stamp: "Formatted at 2026-08-09 09:35:16 on dist-test-slave-7kzw"
server_key: "55067c0f8f10256574c84eefa161ed53"
server_key_iv: "47ae416abbdb75233353503d512d9ef4"
server_key_version: "encryptionkey@0"
I20260809 09:35:16.671682 8745 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.004s
I20260809 09:35:16.674274 8772 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 09:35:16.675848 8745 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 09:35:16.675985 8745 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786267941944493-20982-0/minicluster-data/master-0/data,/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786267941944493-20982-0/minicluster-data/master-0/wal
uuid: "72f5995bd52b4bae956af3750631ef08"
format_stamp: "Formatted at 2026-08-09 09:35:16 on dist-test-slave-7kzw"
server_key: "55067c0f8f10256574c84eefa161ed53"
server_key_iv: "47ae416abbdb75233353503d512d9ef4"
server_key_version: "encryptionkey@0"
I20260809 09:35:16.676137 8745 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786267941944493-20982-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786267941944493-20982-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786267941944493-20982-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 09:35:16.693373 8745 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 09:35:16.696913 8745 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 09:35:16.697140 8745 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 09:35:16.705267 8745 rpc_server.cc:307] RPC server started. Bound to: 127.20.125.190:33057
I20260809 09:35:16.705300 8846 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.125.190:33057 every 8 connection(s)
I20260809 09:35:16.706302 8745 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786267941944493-20982-0/minicluster-data/master-0/data/info.pb
I20260809 09:35:16.709338 8848 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 09:35:16.715333 8848 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 72f5995bd52b4bae956af3750631ef08: Bootstrap starting.
I20260809 09:35:16.716277 20982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu as pid 8745
I20260809 09:35:16.716404 20982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786267941944493-20982-0/minicluster-data/master-0/wal/instance
I20260809 09:35:16.716679 20982 external_mini_cluster.cc:1428] Setting key 7f2c5625a53a0f4f5ee264c58b4bc779
I20260809 09:35:16.718251 8848 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 72f5995bd52b4bae956af3750631ef08: Neither blocks nor log segments found. Creating new log.
I20260809 09:35:16.719122 8848 log.cc:824] T 00000000000000000000000000000000 P 72f5995bd52b4bae956af3750631ef08: Log is configured to *not* fsync() on all Append() calls
I20260809 09:35:16.721387 8848 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 72f5995bd52b4bae956af3750631ef08: No bootstrap required, opened a new log
I20260809 09:35:16.725718 8848 raft_consensus.cc:348] T 00000000000000000000000000000000 P 72f5995bd52b4bae956af3750631ef08 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72f5995bd52b4bae956af3750631ef08" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 33057 } }
I20260809 09:35:16.725961 8848 raft_consensus.cc:374] T 00000000000000000000000000000000 P 72f5995bd52b4bae956af3750631ef08 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 09:35:16.726001 8848 raft_consensus.cc:729] T 00000000000000000000000000000000 P 72f5995bd52b4bae956af3750631ef08 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 72f5995bd52b4bae956af3750631ef08, State: Initialized, Role: FOLLOWER
I20260809 09:35:16.726585 8848 consensus_queue.cc:260] T 00000000000000000000000000000000 P 72f5995bd52b4bae956af3750631ef08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72f5995bd52b4bae956af3750631ef08" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 33057 } }
I20260809 09:35:16.726756 8848 raft_consensus.cc:388] T 00000000000000000000000000000000 P 72f5995bd52b4bae956af3750631ef08 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 09:35:16.726832 8848 raft_consensus.cc:482] T 00000000000000000000000000000000 P 72f5995bd52b4bae956af3750631ef08 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
Aug 09 09:35:16 dist-test-slave-7kzw krb5kdc[8729](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268116, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.125.190@KRBTEST.COM
I20260809 09:35:16.726938 8848 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 72f5995bd52b4bae956af3750631ef08 [term 0 FOLLOWER]: Advancing to term 1
I20260809 09:35:16.728138 8848 raft_consensus.cc:504] T 00000000000000000000000000000000 P 72f5995bd52b4bae956af3750631ef08 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72f5995bd52b4bae956af3750631ef08" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 33057 } }
I20260809 09:35:16.728544 8848 leader_election.cc:302] T 00000000000000000000000000000000 P 72f5995bd52b4bae956af3750631ef08 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 72f5995bd52b4bae956af3750631ef08; no voters:
I20260809 09:35:16.728946 8848 leader_election.cc:288] T 00000000000000000000000000000000 P 72f5995bd52b4bae956af3750631ef08 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 09:35:16.729316 8859 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 72f5995bd52b4bae956af3750631ef08 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 09:35:16.729651 8859 raft_consensus.cc:686] T 00000000000000000000000000000000 P 72f5995bd52b4bae956af3750631ef08 [term 1 LEADER]: Becoming Leader. State: Replica: 72f5995bd52b4bae956af3750631ef08, State: Running, Role: LEADER
I20260809 09:35:16.729996 8859 consensus_queue.cc:237] T 00000000000000000000000000000000 P 72f5995bd52b4bae956af3750631ef08 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72f5995bd52b4bae956af3750631ef08" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 33057 } }
I20260809 09:35:16.730441 8848 sys_catalog.cc:564] T 00000000000000000000000000000000 P 72f5995bd52b4bae956af3750631ef08 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 09:35:16.731343 8859 sys_catalog.cc:455] T 00000000000000000000000000000000 P 72f5995bd52b4bae956af3750631ef08 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 72f5995bd52b4bae956af3750631ef08. Latest consensus state: current_term: 1 leader_uuid: "72f5995bd52b4bae956af3750631ef08" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72f5995bd52b4bae956af3750631ef08" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 33057 } } }
I20260809 09:35:16.731467 8859 sys_catalog.cc:458] T 00000000000000000000000000000000 P 72f5995bd52b4bae956af3750631ef08 [sys.catalog]: This master's current role is: LEADER
I20260809 09:35:16.732661 8860 sys_catalog.cc:455] T 00000000000000000000000000000000 P 72f5995bd52b4bae956af3750631ef08 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "72f5995bd52b4bae956af3750631ef08" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72f5995bd52b4bae956af3750631ef08" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 33057 } } }
I20260809 09:35:16.733476 8860 sys_catalog.cc:458] T 00000000000000000000000000000000 P 72f5995bd52b4bae956af3750631ef08 [sys.catalog]: This master's current role is: LEADER
I20260809 09:35:16.734283 8856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:35:16.717850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43892 (local address 127.20.125.190:33057)
0809 09:35:16.718345 (+ 495us) server_negotiation.cc:206] Beginning negotiation
0809 09:35:16.718360 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 09:35:16.718425 (+ 65us) server_negotiation.cc:407] Connection header received
0809 09:35:16.719125 (+ 700us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:35:16.719145 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:35:16.719546 (+ 401us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:35:16.719880 (+ 334us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:35:16.720849 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:35:16.722347 (+ 1498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:35:16.723352 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:35:16.724138 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:35:16.724367 (+ 229us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:35:16.727716 (+ 3349us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:35:16.727722 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:35:16.727737 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:35:16.727768 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:35:16.730098 (+ 2330us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:35:16.730669 (+ 571us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:35:16.730675 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:35:16.730681 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:35:16.730830 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:35:16.731282 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:35:16.731285 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:35:16.731287 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:35:16.731833 (+ 546us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:35:16.732078 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0809 09:35:16.733117 (+ 1039us) server_negotiation.cc:299] Negotiation successful
0809 09:35:16.733449 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":345,"thread_start_us":163,"threads_started":1}
W20260809 09:35:16.736497 8883 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 72f5995bd52b4bae956af3750631ef08: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 09:35:16.736588 8883 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 09:35:16.736692 8870 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 09:35:16.737658 20982 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 09:35:16.737735 8870 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 09:35:16.743296 8870 catalog_manager.cc:1347] Generated new cluster ID: c88ece37590c43f5a7bae265e22cf57b
I20260809 09:35:16.743378 8870 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 09:35:16.748845 8870 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 09:35:16.750090 8870 catalog_manager.cc:1504] Loading token signing keys...
I20260809 09:35:16.768637 8870 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 72f5995bd52b4bae956af3750631ef08: Generated new TSK 0
I20260809 09:35:16.769539 8870 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 09:35:16.794201 20982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu with pid 8745
2026-08-09T09:35:16Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1677 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 09:35:16 dist-test-slave-7kzw krb5kdc[8892](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 09:35:16 dist-test-slave-7kzw krb5kdc[8892](info): set up 2 sockets
Aug 09 09:35:16 dist-test-slave-7kzw krb5kdc[8892](info): commencing operation
krb5kdc: starting...
W20260809 09:35:18.106846 20982 mini_kdc.cc:121] Time spent starting KDC: real 1.284s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/krb5kdc/test-user.keytab.
Aug 09 09:35:18 dist-test-slave-7kzw krb5kdc[8892](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268118, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T09:35:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T09:35:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.125.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.125.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.125.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.190.keytab.
Entry for principal kudu/127.20.125.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.190.keytab.
I20260809 09:35:18.239370 20982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
/tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.125.190:36897
--webserver_interface=127.20.125.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.190.keytab
--principal=kudu/127.20.125.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.125.148:40667
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.125.190:36897
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 09:35:18.353502 8913 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 09:35:18.353760 8913 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 09:35:18.353801 8913 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 09:35:18.357790 8913 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 09:35:18.357851 8913 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 09:35:18.357870 8913 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 09:35:18.357888 8913 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 09:35:18.357903 8913 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 09:35:18.363125 8913 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.125.148:40667
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.125.190:36897
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.190.keytab
--principal=kudu/127.20.125.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.125.190:36897
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.125.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8913
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 08:38:55 UTC on ccycloud.quasar-njjleu.root.comops.site
build id 4575935
I20260809 09:35:18.364100 8913 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 09:35:18.365072 8913 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 09:35:18.374029 8924 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:35:18.373996 8923 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:35:18.374195 8913 server_base.cc:1034] running on GCE node
W20260809 09:35:18.374010 8926 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:35:18.375041 8913 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 09:35:18.376124 8913 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 09:35:18.377303 8913 hybrid_clock.cc:648] HybridClock initialized: now 1786268118377279 us; error 33 us; skew 500 ppm
Aug 09 09:35:18 dist-test-slave-7kzw krb5kdc[8892](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268118, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 09:35:18.380228 8913 init.cc:377] Logged in from keytab as kudu/127.20.125.190@KRBTEST.COM (short username kudu)
I20260809 09:35:18.381093 8913 webserver.cc:460] Webserver started at http://127.20.125.190:34735/ using document root <none> and password file <none>
I20260809 09:35:18.381631 8913 fs_manager.cc:359] Metadata directory not provided
I20260809 09:35:18.381696 8913 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 09:35:18.381871 8913 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 09:35:18.383998 8913 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/minicluster-data/master-0/data/instance:
uuid: "7a385d41008d4b7aa6e916cf8f8ab796"
format_stamp: "Formatted at 2026-08-09 09:35:18 on dist-test-slave-7kzw"
server_key: "66e740de229301182a61292bfc7e6ea5"
server_key_iv: "714c92229791b2dc8ff32aab8064070f"
server_key_version: "encryptionkey@0"
I20260809 09:35:18.384568 8913 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/minicluster-data/master-0/wal/instance:
uuid: "7a385d41008d4b7aa6e916cf8f8ab796"
format_stamp: "Formatted at 2026-08-09 09:35:18 on dist-test-slave-7kzw"
server_key: "66e740de229301182a61292bfc7e6ea5"
server_key_iv: "714c92229791b2dc8ff32aab8064070f"
server_key_version: "encryptionkey@0"
I20260809 09:35:18.388522 8913 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 09:35:18.391086 8935 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 09:35:18.392455 8913 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 09:35:18.392647 8913 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/minicluster-data/master-0/data,/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/minicluster-data/master-0/wal
uuid: "7a385d41008d4b7aa6e916cf8f8ab796"
format_stamp: "Formatted at 2026-08-09 09:35:18 on dist-test-slave-7kzw"
server_key: "66e740de229301182a61292bfc7e6ea5"
server_key_iv: "714c92229791b2dc8ff32aab8064070f"
server_key_version: "encryptionkey@0"
I20260809 09:35:18.392766 8913 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 09:35:18.411470 8913 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 09:35:18.414775 8913 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 09:35:18.415009 8913 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 09:35:18.422716 8913 rpc_server.cc:307] RPC server started. Bound to: 127.20.125.190:36897
I20260809 09:35:18.422998 9016 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.125.190:36897 every 8 connection(s)
I20260809 09:35:18.423792 8913 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/minicluster-data/master-0/data/info.pb
I20260809 09:35:18.426311 20982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu as pid 8913
I20260809 09:35:18.426456 20982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/minicluster-data/master-0/wal/instance
I20260809 09:35:18.426752 9018 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 09:35:18.426784 20982 external_mini_cluster.cc:1428] Setting key 4ccd6af408b92b32004b0301d654448f
I20260809 09:35:18.434036 9018 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7a385d41008d4b7aa6e916cf8f8ab796: Bootstrap starting.
Aug 09 09:35:18 dist-test-slave-7kzw krb5kdc[8892](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268118, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.125.190@KRBTEST.COM
I20260809 09:35:18.436904 9018 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7a385d41008d4b7aa6e916cf8f8ab796: Neither blocks nor log segments found. Creating new log.
I20260809 09:35:18.437703 9018 log.cc:824] T 00000000000000000000000000000000 P 7a385d41008d4b7aa6e916cf8f8ab796: Log is configured to *not* fsync() on all Append() calls
I20260809 09:35:18.439654 9018 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7a385d41008d4b7aa6e916cf8f8ab796: No bootstrap required, opened a new log
I20260809 09:35:18.442111 9021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:35:18.428188 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50260 (local address 127.20.125.190:36897)
0809 09:35:18.428631 (+ 443us) server_negotiation.cc:206] Beginning negotiation
0809 09:35:18.428654 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0809 09:35:18.428692 (+ 38us) server_negotiation.cc:407] Connection header received
0809 09:35:18.429536 (+ 844us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:35:18.429558 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:35:18.429945 (+ 387us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:35:18.430237 (+ 292us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:35:18.431467 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:35:18.432520 (+ 1053us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:35:18.433716 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:35:18.434243 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:35:18.434374 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:35:18.437261 (+ 2887us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:35:18.437266 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:35:18.437278 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:35:18.437305 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:35:18.439239 (+ 1934us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:35:18.439747 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:35:18.439755 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:35:18.439763 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:35:18.439897 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:35:18.440204 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:35:18.440208 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:35:18.440209 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:35:18.440601 (+ 392us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:35:18.440823 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0809 09:35:18.441123 (+ 300us) server_negotiation.cc:299] Negotiation successful
0809 09:35:18.441409 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":291,"thread_start_us":153,"threads_started":1}
I20260809 09:35:18.443466 9018 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7a385d41008d4b7aa6e916cf8f8ab796 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a385d41008d4b7aa6e916cf8f8ab796" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 36897 } }
I20260809 09:35:18.443724 9018 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7a385d41008d4b7aa6e916cf8f8ab796 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 09:35:18.443809 9018 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7a385d41008d4b7aa6e916cf8f8ab796 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a385d41008d4b7aa6e916cf8f8ab796, State: Initialized, Role: FOLLOWER
I20260809 09:35:18.444240 9018 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7a385d41008d4b7aa6e916cf8f8ab796 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a385d41008d4b7aa6e916cf8f8ab796" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 36897 } }
I20260809 09:35:18.444432 9018 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7a385d41008d4b7aa6e916cf8f8ab796 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 09:35:18.444507 9018 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7a385d41008d4b7aa6e916cf8f8ab796 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 09:35:18.444607 9018 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7a385d41008d4b7aa6e916cf8f8ab796 [term 0 FOLLOWER]: Advancing to term 1
I20260809 09:35:18.445480 9018 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7a385d41008d4b7aa6e916cf8f8ab796 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a385d41008d4b7aa6e916cf8f8ab796" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 36897 } }
I20260809 09:35:18.445813 9018 leader_election.cc:302] T 00000000000000000000000000000000 P 7a385d41008d4b7aa6e916cf8f8ab796 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7a385d41008d4b7aa6e916cf8f8ab796; no voters:
I20260809 09:35:18.446069 9018 leader_election.cc:288] T 00000000000000000000000000000000 P 7a385d41008d4b7aa6e916cf8f8ab796 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 09:35:18.446221 9028 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7a385d41008d4b7aa6e916cf8f8ab796 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 09:35:18.446462 9028 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7a385d41008d4b7aa6e916cf8f8ab796 [term 1 LEADER]: Becoming Leader. State: Replica: 7a385d41008d4b7aa6e916cf8f8ab796, State: Running, Role: LEADER
I20260809 09:35:18.446789 9028 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7a385d41008d4b7aa6e916cf8f8ab796 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a385d41008d4b7aa6e916cf8f8ab796" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 36897 } }
I20260809 09:35:18.447512 9018 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7a385d41008d4b7aa6e916cf8f8ab796 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 09:35:18.448688 9030 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7a385d41008d4b7aa6e916cf8f8ab796 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7a385d41008d4b7aa6e916cf8f8ab796" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a385d41008d4b7aa6e916cf8f8ab796" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 36897 } } }
I20260809 09:35:18.448689 9031 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7a385d41008d4b7aa6e916cf8f8ab796 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7a385d41008d4b7aa6e916cf8f8ab796. Latest consensus state: current_term: 1 leader_uuid: "7a385d41008d4b7aa6e916cf8f8ab796" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a385d41008d4b7aa6e916cf8f8ab796" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 36897 } } }
I20260809 09:35:18.448855 9030 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7a385d41008d4b7aa6e916cf8f8ab796 [sys.catalog]: This master's current role is: LEADER
I20260809 09:35:18.448913 9031 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7a385d41008d4b7aa6e916cf8f8ab796 [sys.catalog]: This master's current role is: LEADER
I20260809 09:35:18.449293 9039 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 09:35:18.452169 9039 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 09:35:18.458073 9039 catalog_manager.cc:1347] Generated new cluster ID: 66f7cc45a5b34d3dbcabda2d042fdf00
I20260809 09:35:18.458166 9039 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 09:35:18.472756 9039 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 09:35:18.473671 9039 catalog_manager.cc:1504] Loading token signing keys...
I20260809 09:35:18.484915 9039 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7a385d41008d4b7aa6e916cf8f8ab796: Generated new TSK 0
I20260809 09:35:18.485602 9039 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.20.125.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.125.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.125.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.129.keytab.
Entry for principal kudu/127.20.125.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.129.keytab.
I20260809 09:35:18.519479 20982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
/tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.125.129:0
--local_ip_for_outbound_sockets=127.20.125.129
--webserver_interface=127.20.125.129
--webserver_port=0
--tserver_master_addrs=127.20.125.190:36897
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.129.keytab
--principal=kudu/127.20.125.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.125.148:40667
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 09:35:18.641335 9061 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 09:35:18.641611 9061 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 09:35:18.641655 9061 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 09:35:18.646299 9061 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 09:35:18.646363 9061 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 09:35:18.646421 9061 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.125.129
I20260809 09:35:18.652035 9061 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.125.148:40667
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.129.keytab
--principal=kudu/127.20.125.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.125.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.125.129
--webserver_port=0
--tserver_master_addrs=127.20.125.190:36897
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9061
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.125.129
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 08:38:55 UTC on ccycloud.quasar-njjleu.root.comops.site
build id 4575935
I20260809 09:35:18.653115 9061 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 09:35:18.654160 9061 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 09:35:18.664544 9068 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:35:18.664645 9069 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:35:18.664608 9061 server_base.cc:1034] running on GCE node
W20260809 09:35:18.664867 9071 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:35:18.665342 9061 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 09:35:18.665992 9061 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 09:35:18.667145 9061 hybrid_clock.cc:648] HybridClock initialized: now 1786268118667111 us; error 48 us; skew 500 ppm
Aug 09 09:35:18 dist-test-slave-7kzw krb5kdc[8892](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268118, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 09:35:18.670208 9061 init.cc:377] Logged in from keytab as kudu/127.20.125.129@KRBTEST.COM (short username kudu)
I20260809 09:35:18.671187 9061 webserver.cc:460] Webserver started at http://127.20.125.129:37749/ using document root <none> and password file <none>
I20260809 09:35:18.671733 9061 fs_manager.cc:359] Metadata directory not provided
I20260809 09:35:18.671795 9061 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 09:35:18.671965 9061 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 09:35:18.673970 9061 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/minicluster-data/ts-0/data/instance:
uuid: "2128f08f594542018da9bd8141d982c4"
format_stamp: "Formatted at 2026-08-09 09:35:18 on dist-test-slave-7kzw"
server_key: "96bc8132391916b37f5a364cd436a105"
server_key_iv: "a83b3a7767b9988e4221b52fc64866b3"
server_key_version: "encryptionkey@0"
I20260809 09:35:18.674595 9061 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/minicluster-data/ts-0/wal/instance:
uuid: "2128f08f594542018da9bd8141d982c4"
format_stamp: "Formatted at 2026-08-09 09:35:18 on dist-test-slave-7kzw"
server_key: "96bc8132391916b37f5a364cd436a105"
server_key_iv: "a83b3a7767b9988e4221b52fc64866b3"
server_key_version: "encryptionkey@0"
I20260809 09:35:18.678745 9061 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260809 09:35:18.681265 9081 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 09:35:18.682617 9061 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 09:35:18.682744 9061 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/minicluster-data/ts-0/data,/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/minicluster-data/ts-0/wal
uuid: "2128f08f594542018da9bd8141d982c4"
format_stamp: "Formatted at 2026-08-09 09:35:18 on dist-test-slave-7kzw"
server_key: "96bc8132391916b37f5a364cd436a105"
server_key_iv: "a83b3a7767b9988e4221b52fc64866b3"
server_key_version: "encryptionkey@0"
I20260809 09:35:18.682853 9061 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 09:35:18.709455 9061 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 09:35:18.713411 9061 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 09:35:18.713699 9061 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 09:35:18.714407 9061 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 09:35:18.715411 9061 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 09:35:18.715474 9061 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 09:35:18.715557 9061 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 09:35:18.715595 9061 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 09:35:18.725605 9061 rpc_server.cc:307] RPC server started. Bound to: 127.20.125.129:33947
I20260809 09:35:18.725641 9248 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.125.129:33947 every 8 connection(s)
I20260809 09:35:18.726722 9061 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/minicluster-data/ts-0/data/info.pb
I20260809 09:35:18.727171 20982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu as pid 9061
I20260809 09:35:18.727370 20982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/minicluster-data/ts-0/wal/instance
I20260809 09:35:18.727773 20982 external_mini_cluster.cc:1428] Setting key bc96ab1813333c9955701c66fe1c8b2f
Aug 09 09:35:18 dist-test-slave-7kzw krb5kdc[8892](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268118, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.129@KRBTEST.COM for kudu/127.20.125.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.20.125.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.125.130@KRBTEST.COM" created.
I20260809 09:35:18.742830 9021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:35:18.728857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.129:54273 (local address 127.20.125.190:36897)
0809 09:35:18.729021 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0809 09:35:18.729027 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 09:35:18.730243 (+ 1216us) server_negotiation.cc:407] Connection header received
0809 09:35:18.731727 (+ 1484us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:35:18.731730 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:35:18.731785 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:35:18.731866 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:35:18.733047 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:35:18.733667 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:35:18.734891 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:35:18.735128 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:35:18.735246 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:35:18.738363 (+ 3117us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:35:18.738366 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:35:18.738368 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:35:18.738400 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:35:18.740060 (+ 1660us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:35:18.740851 (+ 791us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:35:18.740854 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:35:18.740856 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:35:18.740943 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:35:18.741347 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:35:18.741350 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:35:18.741352 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:35:18.741656 (+ 304us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:35:18.741776 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0809 09:35:18.742411 (+ 635us) server_negotiation.cc:299] Negotiation successful
0809 09:35:18.742610 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260809 09:35:18.743923 9255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:35:18.729389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:36897 (local address 127.20.125.129:54273)
0809 09:35:18.730072 (+ 683us) negotiation.cc:107] Waiting for socket to connect
0809 09:35:18.730120 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0809 09:35:18.731508 (+ 1388us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:35:18.732041 (+ 533us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:35:18.732048 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:35:18.732420 (+ 372us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:35:18.732896 (+ 476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:35:18.732910 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:35:18.733812 (+ 902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:35:18.733820 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:35:18.734765 (+ 945us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:35:18.734774 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:35:18.735302 (+ 528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:35:18.735306 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:35:18.735492 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:35:18.736126 (+ 634us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 09:35:18.736144 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:35:18.738147 (+ 2003us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 09:35:18.740219 (+ 2072us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:35:18.740225 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:35:18.740239 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:35:18.740706 (+ 467us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:35:18.741065 (+ 359us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:35:18.741068 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:35:18.741069 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:35:18.741219 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:35:18.741838 (+ 619us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 09:35:18.741845 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 09:35:18.742102 (+ 257us) client_negotiation.cc:769] Sending connection context
0809 09:35:18.742308 (+ 206us) client_negotiation.cc:240] Negotiation successful
0809 09:35:18.742742 (+ 434us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":413,"thread_start_us":164,"threads_started":1}
I20260809 09:35:18.745347 9250 heartbeater.cc:347] Connected to a master server at 127.20.125.190:36897
I20260809 09:35:18.745661 9250 heartbeater.cc:464] Registering TS with master...
I20260809 09:35:18.746342 9250 heartbeater.cc:511] Master 127.20.125.190:36897 requested a full tablet report, sending...
I20260809 09:35:18.748474 8964 ts_manager.cc:194] Registered new tserver with Master: 2128f08f594542018da9bd8141d982c4 (127.20.125.129:33947)
I20260809 09:35:18.749868 8964 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.125.129@KRBTEST.COM'} at 127.20.125.129:54273
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.125.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.130.keytab.
Entry for principal kudu/127.20.125.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.130.keytab.
I20260809 09:35:18.763455 20982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
/tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.125.130:0
--local_ip_for_outbound_sockets=127.20.125.130
--webserver_interface=127.20.125.130
--webserver_port=0
--tserver_master_addrs=127.20.125.190:36897
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.130.keytab
--principal=kudu/127.20.125.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.125.148:40667
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 09:35:18.880645 9260 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 09:35:18.880970 9260 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 09:35:18.881052 9260 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 09:35:18.885087 9260 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 09:35:18.885192 9260 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 09:35:18.885288 9260 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.125.130
I20260809 09:35:18.890836 9260 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.125.148:40667
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.130.keytab
--principal=kudu/127.20.125.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.125.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.125.130
--webserver_port=0
--tserver_master_addrs=127.20.125.190:36897
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9260
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.125.130
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 08:38:55 UTC on ccycloud.quasar-njjleu.root.comops.site
build id 4575935
I20260809 09:35:18.892019 9260 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 09:35:18.893119 9260 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 09:35:18.903674 9260 server_base.cc:1034] running on GCE node
W20260809 09:35:18.903580 9272 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:35:18.903584 9267 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:35:18.903599 9268 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:35:18.904383 9260 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 09:35:18.905082 9260 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 09:35:18.906301 9260 hybrid_clock.cc:648] HybridClock initialized: now 1786268118906257 us; error 55 us; skew 500 ppm
Aug 09 09:35:18 dist-test-slave-7kzw krb5kdc[8892](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268118, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 09:35:18.909560 9260 init.cc:377] Logged in from keytab as kudu/127.20.125.130@KRBTEST.COM (short username kudu)
I20260809 09:35:18.910694 9260 webserver.cc:460] Webserver started at http://127.20.125.130:34923/ using document root <none> and password file <none>
I20260809 09:35:18.911312 9260 fs_manager.cc:359] Metadata directory not provided
I20260809 09:35:18.911409 9260 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 09:35:18.911662 9260 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 09:35:18.914206 9260 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/minicluster-data/ts-1/data/instance:
uuid: "c0b76307d73b4cfdba69b55655d1b26d"
format_stamp: "Formatted at 2026-08-09 09:35:18 on dist-test-slave-7kzw"
server_key: "50584686f8518c032d7ff83ee5686320"
server_key_iv: "ce09d747ffff61d08f59d38ae4903162"
server_key_version: "encryptionkey@0"
I20260809 09:35:18.914822 9260 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/minicluster-data/ts-1/wal/instance:
uuid: "c0b76307d73b4cfdba69b55655d1b26d"
format_stamp: "Formatted at 2026-08-09 09:35:18 on dist-test-slave-7kzw"
server_key: "50584686f8518c032d7ff83ee5686320"
server_key_iv: "ce09d747ffff61d08f59d38ae4903162"
server_key_version: "encryptionkey@0"
I20260809 09:35:18.919025 9260 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260809 09:35:18.921835 9284 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 09:35:18.923296 9260 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 09:35:18.923439 9260 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/minicluster-data/ts-1/data,/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/minicluster-data/ts-1/wal
uuid: "c0b76307d73b4cfdba69b55655d1b26d"
format_stamp: "Formatted at 2026-08-09 09:35:18 on dist-test-slave-7kzw"
server_key: "50584686f8518c032d7ff83ee5686320"
server_key_iv: "ce09d747ffff61d08f59d38ae4903162"
server_key_version: "encryptionkey@0"
I20260809 09:35:18.923568 9260 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 09:35:18.950205 9260 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 09:35:18.953989 9260 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 09:35:18.954259 9260 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 09:35:18.955013 9260 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 09:35:18.956077 9260 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 09:35:18.956138 9260 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 09:35:18.956209 9260 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 09:35:18.956248 9260 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 09:35:18.965602 9260 rpc_server.cc:307] RPC server started. Bound to: 127.20.125.130:46209
I20260809 09:35:18.965639 9443 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.125.130:46209 every 8 connection(s)
I20260809 09:35:18.966580 9260 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/minicluster-data/ts-1/data/info.pb
I20260809 09:35:18.971939 20982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu as pid 9260
I20260809 09:35:18.972085 20982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/minicluster-data/ts-1/wal/instance
I20260809 09:35:18.972383 20982 external_mini_cluster.cc:1428] Setting key 7a726cacd27ba6290755d214cf42490a
Aug 09 09:35:18 dist-test-slave-7kzw krb5kdc[8892](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268118, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.130@KRBTEST.COM for kudu/127.20.125.190@KRBTEST.COM
I20260809 09:35:18.982096 9021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:35:18.968533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.130:37637 (local address 127.20.125.190:36897)
0809 09:35:18.968681 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0809 09:35:18.968685 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 09:35:18.969562 (+ 877us) server_negotiation.cc:407] Connection header received
0809 09:35:18.971173 (+ 1611us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:35:18.971176 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:35:18.971228 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:35:18.971311 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:35:18.972920 (+ 1609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:35:18.973530 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:35:18.975153 (+ 1623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:35:18.975394 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:35:18.975515 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:35:18.978371 (+ 2856us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:35:18.978374 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:35:18.978376 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:35:18.978401 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:35:18.979913 (+ 1512us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:35:18.980460 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:35:18.980464 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:35:18.980465 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:35:18.980559 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:35:18.980927 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:35:18.980931 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:35:18.980933 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:35:18.981129 (+ 196us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:35:18.981248 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0809 09:35:18.981759 (+ 511us) server_negotiation.cc:299] Negotiation successful
0809 09:35:18.981916 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 09:35:18.982993 9446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:35:18.968948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:36897 (local address 127.20.125.130:37637)
0809 09:35:18.969410 (+ 462us) negotiation.cc:107] Waiting for socket to connect
0809 09:35:18.969454 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 09:35:18.970904 (+ 1450us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:35:18.971477 (+ 573us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:35:18.971484 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:35:18.972027 (+ 543us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:35:18.972740 (+ 713us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:35:18.972764 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:35:18.973667 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:35:18.973670 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:35:18.975007 (+ 1337us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:35:18.975023 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:35:18.975571 (+ 548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:35:18.975578 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:35:18.975822 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:35:18.976593 (+ 771us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 09:35:18.976609 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:35:18.978169 (+ 1560us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 09:35:18.980038 (+ 1869us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:35:18.980044 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:35:18.980060 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:35:18.980347 (+ 287us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:35:18.980659 (+ 312us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:35:18.980662 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:35:18.980664 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:35:18.980822 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:35:18.981246 (+ 424us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 09:35:18.981254 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 09:35:18.981510 (+ 256us) client_negotiation.cc:769] Sending connection context
0809 09:35:18.981719 (+ 209us) client_negotiation.cc:240] Negotiation successful
0809 09:35:18.982058 (+ 339us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":274,"thread_start_us":117,"threads_started":1}
WARNING: no policy specified for kudu/127.20.125.131@KRBTEST.COM; defaulting to no policy
I20260809 09:35:18.984246 9444 heartbeater.cc:347] Connected to a master server at 127.20.125.190:36897
I20260809 09:35:18.984524 9444 heartbeater.cc:464] Registering TS with master...
I20260809 09:35:18.985167 9444 heartbeater.cc:511] Master 127.20.125.190:36897 requested a full tablet report, sending...
I20260809 09:35:18.986326 8964 ts_manager.cc:194] Registered new tserver with Master: c0b76307d73b4cfdba69b55655d1b26d (127.20.125.130:46209)
I20260809 09:35:18.986991 8964 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.125.130@KRBTEST.COM'} at 127.20.125.130:37637
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.125.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.125.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.131.keytab.
Entry for principal kudu/127.20.125.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.131.keytab.
I20260809 09:35:19.005915 20982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
/tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.125.131:0
--local_ip_for_outbound_sockets=127.20.125.131
--webserver_interface=127.20.125.131
--webserver_port=0
--tserver_master_addrs=127.20.125.190:36897
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.131.keytab
--principal=kudu/127.20.125.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.125.148:40667
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 09:35:19.121964 9452 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 09:35:19.122251 9452 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 09:35:19.122294 9452 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 09:35:19.126727 9452 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 09:35:19.126816 9452 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 09:35:19.126885 9452 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.125.131
I20260809 09:35:19.133128 9452 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.125.148:40667
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.131.keytab
--principal=kudu/127.20.125.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.125.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.125.131
--webserver_port=0
--tserver_master_addrs=127.20.125.190:36897
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9452
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.125.131
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 08:38:55 UTC on ccycloud.quasar-njjleu.root.comops.site
build id 4575935
I20260809 09:35:19.134296 9452 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 09:35:19.135516 9452 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 09:35:19.146063 9452 server_base.cc:1034] running on GCE node
W20260809 09:35:19.145996 9463 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:35:19.146024 9460 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:35:19.145996 9461 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:35:19.146776 9452 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 09:35:19.147497 9452 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 09:35:19.148694 9452 hybrid_clock.cc:648] HybridClock initialized: now 1786268119148659 us; error 47 us; skew 500 ppm
Aug 09 09:35:19 dist-test-slave-7kzw krb5kdc[8892](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268119, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 09:35:19.152333 9452 init.cc:377] Logged in from keytab as kudu/127.20.125.131@KRBTEST.COM (short username kudu)
I20260809 09:35:19.153350 9452 webserver.cc:460] Webserver started at http://127.20.125.131:41433/ using document root <none> and password file <none>
I20260809 09:35:19.153975 9452 fs_manager.cc:359] Metadata directory not provided
I20260809 09:35:19.154042 9452 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 09:35:19.154285 9452 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 09:35:19.156563 9452 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/minicluster-data/ts-2/data/instance:
uuid: "26f94d9587f546bebd8f7351a6c30140"
format_stamp: "Formatted at 2026-08-09 09:35:19 on dist-test-slave-7kzw"
server_key: "1d155273c6f05f49880d6d2b74b34b6c"
server_key_iv: "e9ff8f13dc172def69b26f94e67bf0bc"
server_key_version: "encryptionkey@0"
I20260809 09:35:19.157155 9452 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/minicluster-data/ts-2/wal/instance:
uuid: "26f94d9587f546bebd8f7351a6c30140"
format_stamp: "Formatted at 2026-08-09 09:35:19 on dist-test-slave-7kzw"
server_key: "1d155273c6f05f49880d6d2b74b34b6c"
server_key_iv: "e9ff8f13dc172def69b26f94e67bf0bc"
server_key_version: "encryptionkey@0"
I20260809 09:35:19.161127 9452 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.001s
I20260809 09:35:19.163985 9476 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 09:35:19.165303 9452 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 09:35:19.165473 9452 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/minicluster-data/ts-2/data,/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/minicluster-data/ts-2/wal
uuid: "26f94d9587f546bebd8f7351a6c30140"
format_stamp: "Formatted at 2026-08-09 09:35:19 on dist-test-slave-7kzw"
server_key: "1d155273c6f05f49880d6d2b74b34b6c"
server_key_iv: "e9ff8f13dc172def69b26f94e67bf0bc"
server_key_version: "encryptionkey@0"
I20260809 09:35:19.165592 9452 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 09:35:19.202538 9452 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 09:35:19.206148 9452 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 09:35:19.206373 9452 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 09:35:19.207053 9452 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 09:35:19.207940 9452 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 09:35:19.208014 9452 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 09:35:19.208086 9452 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 09:35:19.208130 9452 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 09:35:19.217803 9452 rpc_server.cc:307] RPC server started. Bound to: 127.20.125.131:45239
I20260809 09:35:19.217862 9648 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.125.131:45239 every 8 connection(s)
I20260809 09:35:19.218770 9452 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/minicluster-data/ts-2/data/info.pb
I20260809 09:35:19.223604 20982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu as pid 9452
I20260809 09:35:19.223740 20982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/minicluster-data/ts-2/wal/instance
I20260809 09:35:19.224021 20982 external_mini_cluster.cc:1428] Setting key 373f7859ecda7563a22747015e996146
Aug 09 09:35:19 dist-test-slave-7kzw krb5kdc[8892](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268119, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.131@KRBTEST.COM for kudu/127.20.125.190@KRBTEST.COM
I20260809 09:35:19.233597 9021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:35:19.220661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.131:52031 (local address 127.20.125.190:36897)
0809 09:35:19.220923 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0809 09:35:19.220927 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 09:35:19.221696 (+ 769us) server_negotiation.cc:407] Connection header received
0809 09:35:19.223207 (+ 1511us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:35:19.223211 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:35:19.223267 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:35:19.223359 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:35:19.224804 (+ 1445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:35:19.225536 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:35:19.226812 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:35:19.227024 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:35:19.227110 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:35:19.229906 (+ 2796us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:35:19.229909 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:35:19.229912 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:35:19.229935 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:35:19.231627 (+ 1692us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:35:19.232129 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:35:19.232131 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:35:19.232133 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:35:19.232211 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:35:19.232626 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:35:19.232629 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:35:19.232630 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:35:19.232836 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:35:19.232924 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0809 09:35:19.233379 (+ 455us) server_negotiation.cc:299] Negotiation successful
0809 09:35:19.233476 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":157}
I20260809 09:35:19.234416 9652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:35:19.221036 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:36897 (local address 127.20.125.131:52031)
0809 09:35:19.221519 (+ 483us) negotiation.cc:107] Waiting for socket to connect
0809 09:35:19.221564 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 09:35:19.222977 (+ 1413us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:35:19.223558 (+ 581us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:35:19.223565 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:35:19.224023 (+ 458us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:35:19.224585 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:35:19.224603 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:35:19.225675 (+ 1072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:35:19.225679 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:35:19.226704 (+ 1025us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:35:19.226714 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:35:19.227210 (+ 496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:35:19.227215 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:35:19.227414 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:35:19.228074 (+ 660us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 09:35:19.228096 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:35:19.229716 (+ 1620us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 09:35:19.231742 (+ 2026us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:35:19.231751 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:35:19.231762 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:35:19.232028 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:35:19.232330 (+ 302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:35:19.232333 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:35:19.232334 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:35:19.232484 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:35:19.232939 (+ 455us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 09:35:19.232945 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 09:35:19.233169 (+ 224us) client_negotiation.cc:769] Sending connection context
0809 09:35:19.233326 (+ 157us) client_negotiation.cc:240] Negotiation successful
0809 09:35:19.233566 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":285,"thread_start_us":108,"threads_started":1}
I20260809 09:35:19.235607 9649 heartbeater.cc:347] Connected to a master server at 127.20.125.190:36897
I20260809 09:35:19.235873 9649 heartbeater.cc:464] Registering TS with master...
I20260809 09:35:19.236356 9649 heartbeater.cc:511] Master 127.20.125.190:36897 requested a full tablet report, sending...
I20260809 09:35:19.237360 8964 ts_manager.cc:194] Registered new tserver with Master: 26f94d9587f546bebd8f7351a6c30140 (127.20.125.131:45239)
I20260809 09:35:19.238147 8964 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.125.131@KRBTEST.COM'} at 127.20.125.131:52031
I20260809 09:35:19.238389 20982 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 09:35:19.248405 9021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:35:19.240662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50274 (local address 127.20.125.190:36897)
0809 09:35:19.240774 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0809 09:35:19.240777 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 09:35:19.241038 (+ 261us) server_negotiation.cc:407] Connection header received
0809 09:35:19.241173 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:35:19.241176 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:35:19.241214 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:35:19.241294 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:35:19.242028 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:35:19.242581 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:35:19.243438 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:35:19.243659 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:35:19.243776 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:35:19.245128 (+ 1352us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:35:19.245132 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:35:19.245134 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:35:19.245166 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:35:19.246517 (+ 1351us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:35:19.247007 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:35:19.247011 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:35:19.247013 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:35:19.247101 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:35:19.247445 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:35:19.247448 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:35:19.247450 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:35:19.247696 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:35:19.247843 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0809 09:35:19.248088 (+ 245us) server_negotiation.cc:299] Negotiation successful
0809 09:35:19.248210 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260809 09:35:19.251574 8964 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50274:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 09:35:19.254869 8964 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 09:35:19.270874 9669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:35:19.264721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51512 (local address 127.20.125.131:45239)
0809 09:35:19.265521 (+ 800us) server_negotiation.cc:206] Beginning negotiation
0809 09:35:19.265534 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 09:35:19.265563 (+ 29us) server_negotiation.cc:407] Connection header received
0809 09:35:19.265647 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:35:19.265654 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:35:19.265879 (+ 225us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:35:19.266618 (+ 739us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 09:35:19.266752 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:35:19.267660 (+ 908us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:35:19.269330 (+ 1670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:35:19.270140 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:35:19.270315 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:35:19.270553 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0809 09:35:19.270612 (+ 59us) server_negotiation.cc:299] Negotiation successful
0809 09:35:19.270703 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":676,"thread_start_us":169,"threads_started":1}
I20260809 09:35:19.270862 9665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:35:19.264461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.131:45239 (local address 127.0.0.1:51512)
0809 09:35:19.264902 (+ 441us) negotiation.cc:107] Waiting for socket to connect
0809 09:35:19.264954 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0809 09:35:19.265083 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:35:19.266067 (+ 984us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:35:19.266073 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:35:19.266099 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 09:35:19.266286 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:35:19.266298 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:35:19.267801 (+ 1503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:35:19.267807 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:35:19.269198 (+ 1391us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:35:19.269210 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:35:19.270295 (+ 1085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:35:19.270299 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:35:19.270415 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:35:19.270455 (+ 40us) client_negotiation.cc:769] Sending connection context
0809 09:35:19.270583 (+ 128us) client_negotiation.cc:240] Negotiation successful
0809 09:35:19.270719 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":263,"mutex_wait_us":43,"thread_start_us":172,"threads_started":1}
I20260809 09:35:19.271083 9670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:35:19.265359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.129:33947 (local address 127.0.0.1:55724)
0809 09:35:19.266254 (+ 895us) negotiation.cc:107] Waiting for socket to connect
0809 09:35:19.266267 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 09:35:19.266352 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:35:19.266829 (+ 477us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:35:19.266832 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:35:19.266851 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 09:35:19.266944 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:35:19.266951 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:35:19.268280 (+ 1329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:35:19.268283 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:35:19.269400 (+ 1117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:35:19.269411 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:35:19.270674 (+ 1263us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:35:19.270678 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:35:19.270764 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:35:19.270788 (+ 24us) client_negotiation.cc:769] Sending connection context
0809 09:35:19.270858 (+ 70us) client_negotiation.cc:240] Negotiation successful
0809 09:35:19.270934 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":836,"mutex_wait_us":51,"thread_start_us":83,"threads_started":1}
I20260809 09:35:19.271253 9667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:35:19.264500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.130:46209 (local address 127.0.0.1:44702)
0809 09:35:19.265329 (+ 829us) negotiation.cc:107] Waiting for socket to connect
0809 09:35:19.265344 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 09:35:19.265431 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:35:19.266330 (+ 899us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:35:19.266336 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:35:19.266358 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 09:35:19.266501 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:35:19.266510 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:35:19.268218 (+ 1708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:35:19.268222 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:35:19.269675 (+ 1453us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:35:19.269686 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:35:19.270905 (+ 1219us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:35:19.270909 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:35:19.270991 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:35:19.271012 (+ 21us) client_negotiation.cc:769] Sending connection context
0809 09:35:19.271059 (+ 47us) client_negotiation.cc:240] Negotiation successful
0809 09:35:19.271129 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":759,"mutex_wait_us":225,"spinlock_wait_cycles":110464,"thread_start_us":197,"threads_started":1}
I20260809 09:35:19.271512 9674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:35:19.265937 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55724 (local address 127.20.125.129:33947)
0809 09:35:19.266320 (+ 383us) server_negotiation.cc:206] Beginning negotiation
0809 09:35:19.266326 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 09:35:19.266347 (+ 21us) server_negotiation.cc:407] Connection header received
0809 09:35:19.266485 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:35:19.266490 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:35:19.266671 (+ 181us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:35:19.266825 (+ 154us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 09:35:19.267082 (+ 257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:35:19.268182 (+ 1100us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:35:19.269549 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:35:19.270294 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:35:19.270448 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:35:19.270603 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0809 09:35:19.271252 (+ 649us) server_negotiation.cc:299] Negotiation successful
0809 09:35:19.271363 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":276,"thread_start_us":72,"threads_started":1}
I20260809 09:35:19.272091 9672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:35:19.265438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44702 (local address 127.20.125.130:46209)
0809 09:35:19.265837 (+ 399us) server_negotiation.cc:206] Beginning negotiation
0809 09:35:19.265851 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 09:35:19.265876 (+ 25us) server_negotiation.cc:407] Connection header received
0809 09:35:19.265951 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:35:19.265956 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:35:19.266141 (+ 185us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:35:19.266659 (+ 518us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 09:35:19.266844 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:35:19.268055 (+ 1211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:35:19.269820 (+ 1765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:35:19.270767 (+ 947us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:35:19.271640 (+ 873us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:35:19.271776 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0809 09:35:19.271818 (+ 42us) server_negotiation.cc:299] Negotiation successful
0809 09:35:19.271887 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":294,"thread_start_us":193,"threads_started":1}
I20260809 09:35:19.274292 9157 tablet_service.cc:1467] Processing CreateTablet for tablet 35f13e3276144674b6fb07b801a43225 (DEFAULT_TABLE table=test-table [id=734c86a9788f423083b7c0b8f72384a3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 09:35:19.274425 9348 tablet_service.cc:1467] Processing CreateTablet for tablet 35f13e3276144674b6fb07b801a43225 (DEFAULT_TABLE table=test-table [id=734c86a9788f423083b7c0b8f72384a3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 09:35:19.275439 9157 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 35f13e3276144674b6fb07b801a43225. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 09:35:19.275506 9348 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 35f13e3276144674b6fb07b801a43225. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 09:35:19.276934 9555 tablet_service.cc:1467] Processing CreateTablet for tablet 35f13e3276144674b6fb07b801a43225 (DEFAULT_TABLE table=test-table [id=734c86a9788f423083b7c0b8f72384a3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 09:35:19.277958 9555 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 35f13e3276144674b6fb07b801a43225. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 09:35:19.281653 9676 tablet_bootstrap.cc:492] T 35f13e3276144674b6fb07b801a43225 P 2128f08f594542018da9bd8141d982c4: Bootstrap starting.
I20260809 09:35:19.284010 9676 tablet_bootstrap.cc:654] T 35f13e3276144674b6fb07b801a43225 P 2128f08f594542018da9bd8141d982c4: Neither blocks nor log segments found. Creating new log.
I20260809 09:35:19.283984 9678 tablet_bootstrap.cc:492] T 35f13e3276144674b6fb07b801a43225 P c0b76307d73b4cfdba69b55655d1b26d: Bootstrap starting.
I20260809 09:35:19.284854 9676 log.cc:824] T 35f13e3276144674b6fb07b801a43225 P 2128f08f594542018da9bd8141d982c4: Log is configured to *not* fsync() on all Append() calls
I20260809 09:35:19.285161 9679 tablet_bootstrap.cc:492] T 35f13e3276144674b6fb07b801a43225 P 26f94d9587f546bebd8f7351a6c30140: Bootstrap starting.
I20260809 09:35:19.286623 9678 tablet_bootstrap.cc:654] T 35f13e3276144674b6fb07b801a43225 P c0b76307d73b4cfdba69b55655d1b26d: Neither blocks nor log segments found. Creating new log.
I20260809 09:35:19.287134 9676 tablet_bootstrap.cc:492] T 35f13e3276144674b6fb07b801a43225 P 2128f08f594542018da9bd8141d982c4: No bootstrap required, opened a new log
I20260809 09:35:19.287365 9676 ts_tablet_manager.cc:1397] T 35f13e3276144674b6fb07b801a43225 P 2128f08f594542018da9bd8141d982c4: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260809 09:35:19.287431 9678 log.cc:824] T 35f13e3276144674b6fb07b801a43225 P c0b76307d73b4cfdba69b55655d1b26d: Log is configured to *not* fsync() on all Append() calls
I20260809 09:35:19.288180 9679 tablet_bootstrap.cc:654] T 35f13e3276144674b6fb07b801a43225 P 26f94d9587f546bebd8f7351a6c30140: Neither blocks nor log segments found. Creating new log.
I20260809 09:35:19.289378 9678 tablet_bootstrap.cc:492] T 35f13e3276144674b6fb07b801a43225 P c0b76307d73b4cfdba69b55655d1b26d: No bootstrap required, opened a new log
I20260809 09:35:19.289527 9679 log.cc:824] T 35f13e3276144674b6fb07b801a43225 P 26f94d9587f546bebd8f7351a6c30140: Log is configured to *not* fsync() on all Append() calls
I20260809 09:35:19.289584 9678 ts_tablet_manager.cc:1397] T 35f13e3276144674b6fb07b801a43225 P c0b76307d73b4cfdba69b55655d1b26d: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260809 09:35:19.291577 9679 tablet_bootstrap.cc:492] T 35f13e3276144674b6fb07b801a43225 P 26f94d9587f546bebd8f7351a6c30140: No bootstrap required, opened a new log
I20260809 09:35:19.291766 9679 ts_tablet_manager.cc:1397] T 35f13e3276144674b6fb07b801a43225 P 26f94d9587f546bebd8f7351a6c30140: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260809 09:35:19.292066 9676 raft_consensus.cc:348] T 35f13e3276144674b6fb07b801a43225 P 2128f08f594542018da9bd8141d982c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26f94d9587f546bebd8f7351a6c30140" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 45239 } } peers { permanent_uuid: "c0b76307d73b4cfdba69b55655d1b26d" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 46209 } } peers { permanent_uuid: "2128f08f594542018da9bd8141d982c4" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 33947 } }
I20260809 09:35:19.292343 9676 raft_consensus.cc:374] T 35f13e3276144674b6fb07b801a43225 P 2128f08f594542018da9bd8141d982c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 09:35:19.292408 9676 raft_consensus.cc:729] T 35f13e3276144674b6fb07b801a43225 P 2128f08f594542018da9bd8141d982c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2128f08f594542018da9bd8141d982c4, State: Initialized, Role: FOLLOWER
I20260809 09:35:19.292887 9676 consensus_queue.cc:260] T 35f13e3276144674b6fb07b801a43225 P 2128f08f594542018da9bd8141d982c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26f94d9587f546bebd8f7351a6c30140" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 45239 } } peers { permanent_uuid: "c0b76307d73b4cfdba69b55655d1b26d" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 46209 } } peers { permanent_uuid: "2128f08f594542018da9bd8141d982c4" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 33947 } }
I20260809 09:35:19.292850 9678 raft_consensus.cc:348] T 35f13e3276144674b6fb07b801a43225 P c0b76307d73b4cfdba69b55655d1b26d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26f94d9587f546bebd8f7351a6c30140" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 45239 } } peers { permanent_uuid: "c0b76307d73b4cfdba69b55655d1b26d" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 46209 } } peers { permanent_uuid: "2128f08f594542018da9bd8141d982c4" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 33947 } }
I20260809 09:35:19.293066 9678 raft_consensus.cc:374] T 35f13e3276144674b6fb07b801a43225 P c0b76307d73b4cfdba69b55655d1b26d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 09:35:19.293120 9678 raft_consensus.cc:729] T 35f13e3276144674b6fb07b801a43225 P c0b76307d73b4cfdba69b55655d1b26d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c0b76307d73b4cfdba69b55655d1b26d, State: Initialized, Role: FOLLOWER
I20260809 09:35:19.293519 9678 consensus_queue.cc:260] T 35f13e3276144674b6fb07b801a43225 P c0b76307d73b4cfdba69b55655d1b26d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26f94d9587f546bebd8f7351a6c30140" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 45239 } } peers { permanent_uuid: "c0b76307d73b4cfdba69b55655d1b26d" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 46209 } } peers { permanent_uuid: "2128f08f594542018da9bd8141d982c4" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 33947 } }
I20260809 09:35:19.294008 9676 ts_tablet_manager.cc:1428] T 35f13e3276144674b6fb07b801a43225 P 2128f08f594542018da9bd8141d982c4: Time spent starting tablet: real 0.007s user 0.003s sys 0.004s
I20260809 09:35:19.294174 9444 heartbeater.cc:503] Master 127.20.125.190:36897 was elected leader, sending a full tablet report...
I20260809 09:35:19.294375 9250 heartbeater.cc:503] Master 127.20.125.190:36897 was elected leader, sending a full tablet report...
I20260809 09:35:19.295037 9678 ts_tablet_manager.cc:1428] T 35f13e3276144674b6fb07b801a43225 P c0b76307d73b4cfdba69b55655d1b26d: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260809 09:35:19.296391 9679 raft_consensus.cc:348] T 35f13e3276144674b6fb07b801a43225 P 26f94d9587f546bebd8f7351a6c30140 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26f94d9587f546bebd8f7351a6c30140" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 45239 } } peers { permanent_uuid: "c0b76307d73b4cfdba69b55655d1b26d" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 46209 } } peers { permanent_uuid: "2128f08f594542018da9bd8141d982c4" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 33947 } }
I20260809 09:35:19.296676 9679 raft_consensus.cc:374] T 35f13e3276144674b6fb07b801a43225 P 26f94d9587f546bebd8f7351a6c30140 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 09:35:19.296743 9679 raft_consensus.cc:729] T 35f13e3276144674b6fb07b801a43225 P 26f94d9587f546bebd8f7351a6c30140 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 26f94d9587f546bebd8f7351a6c30140, State: Initialized, Role: FOLLOWER
I20260809 09:35:19.297148 9679 consensus_queue.cc:260] T 35f13e3276144674b6fb07b801a43225 P 26f94d9587f546bebd8f7351a6c30140 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26f94d9587f546bebd8f7351a6c30140" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 45239 } } peers { permanent_uuid: "c0b76307d73b4cfdba69b55655d1b26d" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 46209 } } peers { permanent_uuid: "2128f08f594542018da9bd8141d982c4" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 33947 } }
I20260809 09:35:19.297933 9649 heartbeater.cc:503] Master 127.20.125.190:36897 was elected leader, sending a full tablet report...
I20260809 09:35:19.299110 9679 ts_tablet_manager.cc:1428] T 35f13e3276144674b6fb07b801a43225 P 26f94d9587f546bebd8f7351a6c30140: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260809 09:35:19.322363 9684 raft_consensus.cc:482] T 35f13e3276144674b6fb07b801a43225 P 2128f08f594542018da9bd8141d982c4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 09:35:19.322659 9684 raft_consensus.cc:504] T 35f13e3276144674b6fb07b801a43225 P 2128f08f594542018da9bd8141d982c4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26f94d9587f546bebd8f7351a6c30140" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 45239 } } peers { permanent_uuid: "c0b76307d73b4cfdba69b55655d1b26d" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 46209 } } peers { permanent_uuid: "2128f08f594542018da9bd8141d982c4" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 33947 } }
I20260809 09:35:19.323886 9684 leader_election.cc:288] T 35f13e3276144674b6fb07b801a43225 P 2128f08f594542018da9bd8141d982c4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 26f94d9587f546bebd8f7351a6c30140 (127.20.125.131:45239), c0b76307d73b4cfdba69b55655d1b26d (127.20.125.130:46209)
I20260809 09:35:19.328188 9672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:35:19.324316 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.129:55645 (local address 127.20.125.130:46209)
0809 09:35:19.324454 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0809 09:35:19.324458 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 09:35:19.324713 (+ 255us) server_negotiation.cc:407] Connection header received
0809 09:35:19.324907 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:35:19.324910 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:35:19.324964 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:35:19.325054 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 09:35:19.325350 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:35:19.325876 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:35:19.327055 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:35:19.327568 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:35:19.327736 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:35:19.327792 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0809 09:35:19.327975 (+ 183us) server_negotiation.cc:299] Negotiation successful
0809 09:35:19.328063 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260809 09:35:19.328300 9669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:35:19.324268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.129:54327 (local address 127.20.125.131:45239)
0809 09:35:19.324416 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0809 09:35:19.324421 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 09:35:19.324707 (+ 286us) server_negotiation.cc:407] Connection header received
0809 09:35:19.324902 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:35:19.324910 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:35:19.324954 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:35:19.325038 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 09:35:19.325323 (+ 285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:35:19.325828 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:35:19.327082 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:35:19.327651 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:35:19.327789 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:35:19.327863 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0809 09:35:19.328097 (+ 234us) server_negotiation.cc:299] Negotiation successful
0809 09:35:19.328167 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260809 09:35:19.328210 9693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:35:19.324189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.130:46209 (local address 127.20.125.129:55645)
0809 09:35:19.324597 (+ 408us) negotiation.cc:107] Waiting for socket to connect
0809 09:35:19.324610 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 09:35:19.324783 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:35:19.325067 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:35:19.325070 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:35:19.325095 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 09:35:19.325225 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:35:19.325230 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:35:19.326001 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:35:19.326006 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:35:19.326925 (+ 919us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:35:19.326935 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:35:19.327737 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:35:19.327740 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:35:19.327845 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:35:19.327869 (+ 24us) client_negotiation.cc:769] Sending connection context
0809 09:35:19.327943 (+ 74us) client_negotiation.cc:240] Negotiation successful
0809 09:35:19.328015 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":332,"mutex_wait_us":4,"spinlock_wait_cycles":5888,"thread_start_us":146,"threads_started":1}
I20260809 09:35:19.328238 9691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:35:19.324196 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.131:45239 (local address 127.20.125.129:54327)
0809 09:35:19.324599 (+ 403us) negotiation.cc:107] Waiting for socket to connect
0809 09:35:19.324611 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 09:35:19.324781 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:35:19.325050 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:35:19.325053 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:35:19.325069 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 09:35:19.325207 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:35:19.325214 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:35:19.325941 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:35:19.325947 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:35:19.326957 (+ 1010us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:35:19.326967 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:35:19.327864 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:35:19.327867 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:35:19.327974 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:35:19.327992 (+ 18us) client_negotiation.cc:769] Sending connection context
0809 09:35:19.328067 (+ 75us) client_negotiation.cc:240] Negotiation successful
0809 09:35:19.328123 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":347,"mutex_wait_us":83,"spinlock_wait_cycles":41216,"thread_start_us":130,"threads_started":1}
I20260809 09:35:19.329166 9375 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "35f13e3276144674b6fb07b801a43225" candidate_uuid: "2128f08f594542018da9bd8141d982c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c0b76307d73b4cfdba69b55655d1b26d" is_pre_election: true
I20260809 09:35:19.329279 9582 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "35f13e3276144674b6fb07b801a43225" candidate_uuid: "2128f08f594542018da9bd8141d982c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "26f94d9587f546bebd8f7351a6c30140" is_pre_election: true
I20260809 09:35:19.329546 9375 raft_consensus.cc:2454] T 35f13e3276144674b6fb07b801a43225 P c0b76307d73b4cfdba69b55655d1b26d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2128f08f594542018da9bd8141d982c4 in term 0.
I20260809 09:35:19.329627 9582 raft_consensus.cc:2454] T 35f13e3276144674b6fb07b801a43225 P 26f94d9587f546bebd8f7351a6c30140 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2128f08f594542018da9bd8141d982c4 in term 0.
I20260809 09:35:19.330060 9084 leader_election.cc:302] T 35f13e3276144674b6fb07b801a43225 P 2128f08f594542018da9bd8141d982c4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2128f08f594542018da9bd8141d982c4, c0b76307d73b4cfdba69b55655d1b26d; no voters:
I20260809 09:35:19.330348 9684 raft_consensus.cc:2781] T 35f13e3276144674b6fb07b801a43225 P 2128f08f594542018da9bd8141d982c4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 09:35:19.330436 9684 raft_consensus.cc:482] T 35f13e3276144674b6fb07b801a43225 P 2128f08f594542018da9bd8141d982c4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 09:35:19.330509 9684 raft_consensus.cc:3037] T 35f13e3276144674b6fb07b801a43225 P 2128f08f594542018da9bd8141d982c4 [term 0 FOLLOWER]: Advancing to term 1
I20260809 09:35:19.331847 9684 raft_consensus.cc:504] T 35f13e3276144674b6fb07b801a43225 P 2128f08f594542018da9bd8141d982c4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26f94d9587f546bebd8f7351a6c30140" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 45239 } } peers { permanent_uuid: "c0b76307d73b4cfdba69b55655d1b26d" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 46209 } } peers { permanent_uuid: "2128f08f594542018da9bd8141d982c4" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 33947 } }
I20260809 09:35:19.332290 9684 leader_election.cc:288] T 35f13e3276144674b6fb07b801a43225 P 2128f08f594542018da9bd8141d982c4 [CANDIDATE]: Term 1 election: Requested vote from peers 26f94d9587f546bebd8f7351a6c30140 (127.20.125.131:45239), c0b76307d73b4cfdba69b55655d1b26d (127.20.125.130:46209)
I20260809 09:35:19.332608 9375 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "35f13e3276144674b6fb07b801a43225" candidate_uuid: "2128f08f594542018da9bd8141d982c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c0b76307d73b4cfdba69b55655d1b26d"
I20260809 09:35:19.332618 9582 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "35f13e3276144674b6fb07b801a43225" candidate_uuid: "2128f08f594542018da9bd8141d982c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "26f94d9587f546bebd8f7351a6c30140"
I20260809 09:35:19.332753 9375 raft_consensus.cc:3037] T 35f13e3276144674b6fb07b801a43225 P c0b76307d73b4cfdba69b55655d1b26d [term 0 FOLLOWER]: Advancing to term 1
I20260809 09:35:19.332760 9582 raft_consensus.cc:3037] T 35f13e3276144674b6fb07b801a43225 P 26f94d9587f546bebd8f7351a6c30140 [term 0 FOLLOWER]: Advancing to term 1
I20260809 09:35:19.333796 9375 raft_consensus.cc:2454] T 35f13e3276144674b6fb07b801a43225 P c0b76307d73b4cfdba69b55655d1b26d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2128f08f594542018da9bd8141d982c4 in term 1.
I20260809 09:35:19.333838 9582 raft_consensus.cc:2454] T 35f13e3276144674b6fb07b801a43225 P 26f94d9587f546bebd8f7351a6c30140 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2128f08f594542018da9bd8141d982c4 in term 1.
I20260809 09:35:19.334076 9084 leader_election.cc:302] T 35f13e3276144674b6fb07b801a43225 P 2128f08f594542018da9bd8141d982c4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2128f08f594542018da9bd8141d982c4, c0b76307d73b4cfdba69b55655d1b26d; no voters:
I20260809 09:35:19.334259 9684 raft_consensus.cc:2781] T 35f13e3276144674b6fb07b801a43225 P 2128f08f594542018da9bd8141d982c4 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 09:35:19.334543 9684 raft_consensus.cc:686] T 35f13e3276144674b6fb07b801a43225 P 2128f08f594542018da9bd8141d982c4 [term 1 LEADER]: Becoming Leader. State: Replica: 2128f08f594542018da9bd8141d982c4, State: Running, Role: LEADER
I20260809 09:35:19.334925 9684 consensus_queue.cc:237] T 35f13e3276144674b6fb07b801a43225 P 2128f08f594542018da9bd8141d982c4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26f94d9587f546bebd8f7351a6c30140" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 45239 } } peers { permanent_uuid: "c0b76307d73b4cfdba69b55655d1b26d" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 46209 } } peers { permanent_uuid: "2128f08f594542018da9bd8141d982c4" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 33947 } }
I20260809 09:35:19.338063 8960 catalog_manager.cc:5495] T 35f13e3276144674b6fb07b801a43225 P 2128f08f594542018da9bd8141d982c4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2128f08f594542018da9bd8141d982c4 (127.20.125.129). New cstate: current_term: 1 leader_uuid: "2128f08f594542018da9bd8141d982c4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26f94d9587f546bebd8f7351a6c30140" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 45239 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c0b76307d73b4cfdba69b55655d1b26d" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 46209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2128f08f594542018da9bd8141d982c4" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 33947 } health_report { overall_health: HEALTHY } } }
I20260809 09:35:19.356761 9674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:35:19.353185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55740 (local address 127.20.125.129:33947)
0809 09:35:19.353323 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0809 09:35:19.353327 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 09:35:19.353343 (+ 16us) server_negotiation.cc:407] Connection header received
0809 09:35:19.353419 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:35:19.353421 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:35:19.353478 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:35:19.353561 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 09:35:19.353824 (+ 263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:35:19.354323 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:35:19.355193 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:35:19.355400 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:35:19.355479 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:35:19.355782 (+ 303us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 09:35:19.356215 (+ 433us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 09:35:19.356384 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0809 09:35:19.356464 (+ 80us) server_negotiation.cc:299] Negotiation successful
0809 09:35:19.356534 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260809 09:35:19.362743 9582 raft_consensus.cc:1261] T 35f13e3276144674b6fb07b801a43225 P 26f94d9587f546bebd8f7351a6c30140 [term 1 FOLLOWER]: Refusing update from remote peer 2128f08f594542018da9bd8141d982c4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 09:35:19.362710 9375 raft_consensus.cc:1261] T 35f13e3276144674b6fb07b801a43225 P c0b76307d73b4cfdba69b55655d1b26d [term 1 FOLLOWER]: Refusing update from remote peer 2128f08f594542018da9bd8141d982c4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 09:35:19.363483 9694 consensus_queue.cc:1035] T 35f13e3276144674b6fb07b801a43225 P 2128f08f594542018da9bd8141d982c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "26f94d9587f546bebd8f7351a6c30140" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 45239 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 09:35:19.363719 9684 consensus_queue.cc:1035] T 35f13e3276144674b6fb07b801a43225 P 2128f08f594542018da9bd8141d982c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c0b76307d73b4cfdba69b55655d1b26d" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 46209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 09:35:19.371869 9700 mvcc.cc:205] Tried to move back new op lower bound from 7316554216902348800 to 7316554216798605312. Current Snapshot: MvccSnapshot[applied={T|T < 7316554216902348800}]
I20260809 09:35:19.381335 8960 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50274:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 09:35:19.381529 8960 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50274:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 09:35:19.384243 8960 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7a385d41008d4b7aa6e916cf8f8ab796: Sending DeleteTablet for 3 replicas of tablet 35f13e3276144674b6fb07b801a43225
I20260809 09:35:19.385108 9348 tablet_service.cc:1514] Processing DeleteTablet for tablet 35f13e3276144674b6fb07b801a43225 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 09:35:19 UTC) from {username='kudu', principal='kudu/127.20.125.190@KRBTEST.COM'} at 127.0.0.1:44702
I20260809 09:35:19.385108 9157 tablet_service.cc:1514] Processing DeleteTablet for tablet 35f13e3276144674b6fb07b801a43225 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 09:35:19 UTC) from {username='kudu', principal='kudu/127.20.125.190@KRBTEST.COM'} at 127.0.0.1:55724
I20260809 09:35:19.385191 9555 tablet_service.cc:1514] Processing DeleteTablet for tablet 35f13e3276144674b6fb07b801a43225 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 09:35:19 UTC) from {username='kudu', principal='kudu/127.20.125.190@KRBTEST.COM'} at 127.0.0.1:51512
I20260809 09:35:19.385715 9719 tablet_replica.cc:330] T 35f13e3276144674b6fb07b801a43225 P c0b76307d73b4cfdba69b55655d1b26d: stopping tablet replica
I20260809 09:35:19.385838 9721 tablet_replica.cc:330] T 35f13e3276144674b6fb07b801a43225 P 26f94d9587f546bebd8f7351a6c30140: stopping tablet replica
I20260809 09:35:19.385871 20982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu with pid 9061
I20260809 09:35:19.390910 9721 raft_consensus.cc:2229] T 35f13e3276144674b6fb07b801a43225 P 26f94d9587f546bebd8f7351a6c30140 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 09:35:19.391355 9721 raft_consensus.cc:2258] T 35f13e3276144674b6fb07b801a43225 P 26f94d9587f546bebd8f7351a6c30140 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 09:35:19.392359 9719 raft_consensus.cc:2229] T 35f13e3276144674b6fb07b801a43225 P c0b76307d73b4cfdba69b55655d1b26d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 09:35:19.392736 9719 raft_consensus.cc:2258] T 35f13e3276144674b6fb07b801a43225 P c0b76307d73b4cfdba69b55655d1b26d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 09:35:19.394876 9719 ts_tablet_manager.cc:1905] T 35f13e3276144674b6fb07b801a43225 P c0b76307d73b4cfdba69b55655d1b26d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 09:35:19.394860 9721 ts_tablet_manager.cc:1905] T 35f13e3276144674b6fb07b801a43225 P 26f94d9587f546bebd8f7351a6c30140: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 09:35:19.397711 9719 ts_tablet_manager.cc:1918] T 35f13e3276144674b6fb07b801a43225 P c0b76307d73b4cfdba69b55655d1b26d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 09:35:19.397846 9719 log.cc:1196] T 35f13e3276144674b6fb07b801a43225 P c0b76307d73b4cfdba69b55655d1b26d: Deleting WAL directory at /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/minicluster-data/ts-1/wal/wals/35f13e3276144674b6fb07b801a43225
I20260809 09:35:19.398300 9719 ts_tablet_manager.cc:1939] T 35f13e3276144674b6fb07b801a43225 P c0b76307d73b4cfdba69b55655d1b26d: Deleting consensus metadata
I20260809 09:35:19.398447 9721 ts_tablet_manager.cc:1918] T 35f13e3276144674b6fb07b801a43225 P 26f94d9587f546bebd8f7351a6c30140: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 09:35:19.398590 9721 log.cc:1196] T 35f13e3276144674b6fb07b801a43225 P 26f94d9587f546bebd8f7351a6c30140: Deleting WAL directory at /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/minicluster-data/ts-2/wal/wals/35f13e3276144674b6fb07b801a43225
I20260809 09:35:19.398929 9721 ts_tablet_manager.cc:1939] T 35f13e3276144674b6fb07b801a43225 P 26f94d9587f546bebd8f7351a6c30140: Deleting consensus metadata
I20260809 09:35:19.399778 8939 catalog_manager.cc:4841] TS c0b76307d73b4cfdba69b55655d1b26d (127.20.125.130:46209): tablet 35f13e3276144674b6fb07b801a43225 (table test-table [id=734c86a9788f423083b7c0b8f72384a3]) successfully deleted
I20260809 09:35:19.400118 8936 catalog_manager.cc:4841] TS 26f94d9587f546bebd8f7351a6c30140 (127.20.125.131:45239): tablet 35f13e3276144674b6fb07b801a43225 (table test-table [id=734c86a9788f423083b7c0b8f72384a3]) successfully deleted
W20260809 09:35:19.402410 8936 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.20.125.129:33947 (error 108)
I20260809 09:35:19.402590 20982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu with pid 9260
I20260809 09:35:19.404871 9667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:35:19.403576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.129:33947 (local address 127.0.0.1:55750)
0809 09:35:19.404676 (+ 1100us) negotiation.cc:107] Waiting for socket to connect
0809 09:35:19.404760 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.129:33947: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":989}
W20260809 09:35:19.405041 8936 catalog_manager.cc:4584] TS 2128f08f594542018da9bd8141d982c4 (127.20.125.129:33947): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 35f13e3276144674b6fb07b801a43225: Network error: Client connection negotiation failed: client connection to 127.20.125.129:33947: connect: Connection refused (error 111)
I20260809 09:35:19.417342 20982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu with pid 9452
I20260809 09:35:19.429070 20982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu with pid 8913
I20260809 09:35:19.441946 20982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
/tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.125.190:36897
--webserver_interface=127.20.125.190
--webserver_port=34735
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.190.keytab
--principal=kudu/127.20.125.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.125.148:40667
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.125.190:36897
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 09:35:19.556160 9723 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 09:35:19.556468 9723 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 09:35:19.556548 9723 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 09:35:19.560591 9723 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 09:35:19.560679 9723 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 09:35:19.560719 9723 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 09:35:19.560778 9723 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 09:35:19.560812 9723 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 09:35:19.566025 9723 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.125.148:40667
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.20.125.190:36897
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.190.keytab
--principal=kudu/127.20.125.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.125.190:36897
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.125.190
--webserver_port=34735
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9723
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 08:38:55 UTC on ccycloud.quasar-njjleu.root.comops.site
build id 4575935
I20260809 09:35:19.567193 9723 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 09:35:19.568253 9723 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 09:35:19.583422 9740 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:35:19.583415 9736 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:35:19.583673 9723 server_base.cc:1034] running on GCE node
W20260809 09:35:19.583437 9734 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:35:19.584342 9723 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 09:35:19.585531 9723 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 09:35:19.586762 9723 hybrid_clock.cc:648] HybridClock initialized: now 1786268119586746 us; error 28 us; skew 500 ppm
Aug 09 09:35:19 dist-test-slave-7kzw krb5kdc[8892](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268119, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 09:35:19.589798 9723 init.cc:377] Logged in from keytab as kudu/127.20.125.190@KRBTEST.COM (short username kudu)
I20260809 09:35:19.590828 9723 webserver.cc:460] Webserver started at http://127.20.125.190:34735/ using document root <none> and password file <none>
I20260809 09:35:19.591426 9723 fs_manager.cc:359] Metadata directory not provided
I20260809 09:35:19.591548 9723 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 09:35:19.595512 9723 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260809 09:35:19.597765 9747 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 09:35:19.599067 9723 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 09:35:19.599237 9723 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/minicluster-data/master-0/data,/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/minicluster-data/master-0/wal
uuid: "7a385d41008d4b7aa6e916cf8f8ab796"
format_stamp: "Formatted at 2026-08-09 09:35:18 on dist-test-slave-7kzw"
server_key: "66e740de229301182a61292bfc7e6ea5"
server_key_iv: "714c92229791b2dc8ff32aab8064070f"
server_key_version: "encryptionkey@0"
I20260809 09:35:19.599934 9723 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 09:35:19.623962 9723 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 09:35:19.627534 9723 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 09:35:19.627774 9723 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 09:35:19.635797 9723 rpc_server.cc:307] RPC server started. Bound to: 127.20.125.190:36897
I20260809 09:35:19.636102 9818 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.125.190:36897 every 8 connection(s)
I20260809 09:35:19.636981 9723 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/minicluster-data/master-0/data/info.pb
I20260809 09:35:19.639564 20982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu as pid 9723
I20260809 09:35:19.640103 20982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
/tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.125.129:33947
--local_ip_for_outbound_sockets=127.20.125.129
--tserver_master_addrs=127.20.125.190:36897
--webserver_port=37749
--webserver_interface=127.20.125.129
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.129.keytab
--principal=kudu/127.20.125.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.125.148:40667
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 09:35:19.642334 9819 sys_catalog.cc:263] Verifying existing consensus state
I20260809 09:35:19.644017 9819 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7a385d41008d4b7aa6e916cf8f8ab796: Bootstrap starting.
I20260809 09:35:19.663028 9819 log.cc:824] T 00000000000000000000000000000000 P 7a385d41008d4b7aa6e916cf8f8ab796: Log is configured to *not* fsync() on all Append() calls
I20260809 09:35:19.674507 9819 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7a385d41008d4b7aa6e916cf8f8ab796: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 09:35:19.674957 9819 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7a385d41008d4b7aa6e916cf8f8ab796: Bootstrap complete.
I20260809 09:35:19.680912 9819 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7a385d41008d4b7aa6e916cf8f8ab796 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a385d41008d4b7aa6e916cf8f8ab796" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 36897 } }
I20260809 09:35:19.681449 9819 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7a385d41008d4b7aa6e916cf8f8ab796 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a385d41008d4b7aa6e916cf8f8ab796, State: Initialized, Role: FOLLOWER
I20260809 09:35:19.682008 9819 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7a385d41008d4b7aa6e916cf8f8ab796 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a385d41008d4b7aa6e916cf8f8ab796" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 36897 } }
I20260809 09:35:19.682207 9819 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7a385d41008d4b7aa6e916cf8f8ab796 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 09:35:19.682303 9819 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7a385d41008d4b7aa6e916cf8f8ab796 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 09:35:19.682407 9819 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7a385d41008d4b7aa6e916cf8f8ab796 [term 1 FOLLOWER]: Advancing to term 2
I20260809 09:35:19.684971 9819 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7a385d41008d4b7aa6e916cf8f8ab796 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a385d41008d4b7aa6e916cf8f8ab796" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 36897 } }
I20260809 09:35:19.685343 9819 leader_election.cc:302] T 00000000000000000000000000000000 P 7a385d41008d4b7aa6e916cf8f8ab796 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7a385d41008d4b7aa6e916cf8f8ab796; no voters:
I20260809 09:35:19.685722 9819 leader_election.cc:288] T 00000000000000000000000000000000 P 7a385d41008d4b7aa6e916cf8f8ab796 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260809 09:35:19.685859 9825 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7a385d41008d4b7aa6e916cf8f8ab796 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 09:35:19.686149 9825 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7a385d41008d4b7aa6e916cf8f8ab796 [term 2 LEADER]: Becoming Leader. State: Replica: 7a385d41008d4b7aa6e916cf8f8ab796, State: Running, Role: LEADER
I20260809 09:35:19.686506 9825 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7a385d41008d4b7aa6e916cf8f8ab796 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a385d41008d4b7aa6e916cf8f8ab796" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 36897 } }
I20260809 09:35:19.687060 9819 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7a385d41008d4b7aa6e916cf8f8ab796 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 09:35:19.687559 9831 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7a385d41008d4b7aa6e916cf8f8ab796 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7a385d41008d4b7aa6e916cf8f8ab796. Latest consensus state: current_term: 2 leader_uuid: "7a385d41008d4b7aa6e916cf8f8ab796" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a385d41008d4b7aa6e916cf8f8ab796" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 36897 } } }
I20260809 09:35:19.687700 9831 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7a385d41008d4b7aa6e916cf8f8ab796 [sys.catalog]: This master's current role is: LEADER
I20260809 09:35:19.688002 9826 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7a385d41008d4b7aa6e916cf8f8ab796 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "7a385d41008d4b7aa6e916cf8f8ab796" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a385d41008d4b7aa6e916cf8f8ab796" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 36897 } } }
I20260809 09:35:19.688089 9826 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7a385d41008d4b7aa6e916cf8f8ab796 [sys.catalog]: This master's current role is: LEADER
I20260809 09:35:19.688143 9835 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 09:35:19.692924 9835 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 09:35:19.693617 9835 catalog_manager.cc:1259] Loaded cluster ID: 66f7cc45a5b34d3dbcabda2d042fdf00
I20260809 09:35:19.693682 9835 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 09:35:19.695164 9835 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260809 09:35:19.700126 9835 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260809 09:35:19.705924 9835 catalog_manager.cc:1504] Loading token signing keys...
I20260809 09:35:19.706781 9835 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 7a385d41008d4b7aa6e916cf8f8ab796: Loaded TSK: 0
I20260809 09:35:19.708002 9835 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260809 09:35:19.769284 9822 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 09:35:19.769641 9822 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 09:35:19.769737 9822 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 09:35:19.774125 9822 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 09:35:19.774216 9822 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 09:35:19.774309 9822 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.125.129
I20260809 09:35:19.779596 9822 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.125.148:40667
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.129.keytab
--principal=kudu/127.20.125.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.125.129:33947
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.125.129
--webserver_port=37749
--tserver_master_addrs=127.20.125.190:36897
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9822
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.125.129
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 08:38:55 UTC on ccycloud.quasar-njjleu.root.comops.site
build id 4575935
I20260809 09:35:19.780609 9822 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 09:35:19.781646 9822 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 09:35:19.792582 9870 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:35:19.792582 9872 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:35:19.793285 9866 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:35:19.793622 9822 server_base.cc:1034] running on GCE node
I20260809 09:35:19.794077 9822 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 09:35:19.794885 9822 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 09:35:19.796104 9822 hybrid_clock.cc:648] HybridClock initialized: now 1786268119796073 us; error 46 us; skew 500 ppm
Aug 09 09:35:19 dist-test-slave-7kzw krb5kdc[8892](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268119, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 09:35:19.799203 9822 init.cc:377] Logged in from keytab as kudu/127.20.125.129@KRBTEST.COM (short username kudu)
I20260809 09:35:19.800163 9822 webserver.cc:460] Webserver started at http://127.20.125.129:37749/ using document root <none> and password file <none>
I20260809 09:35:19.800772 9822 fs_manager.cc:359] Metadata directory not provided
I20260809 09:35:19.800896 9822 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 09:35:19.805249 9822 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260809 09:35:19.807689 9884 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 09:35:19.809015 9822 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 09:35:19.809170 9822 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/minicluster-data/ts-0/data,/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/minicluster-data/ts-0/wal
uuid: "2128f08f594542018da9bd8141d982c4"
format_stamp: "Formatted at 2026-08-09 09:35:18 on dist-test-slave-7kzw"
server_key: "96bc8132391916b37f5a364cd436a105"
server_key_iv: "a83b3a7767b9988e4221b52fc64866b3"
server_key_version: "encryptionkey@0"
I20260809 09:35:19.809715 9822 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 09:35:19.828382 9822 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 09:35:19.831931 9822 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 09:35:19.832149 9822 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 09:35:19.832804 9822 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 09:35:19.833925 9892 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260809 09:35:19.839646 9822 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260809 09:35:19.839745 9822 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.000s sys 0.000s
I20260809 09:35:19.839851 9822 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260809 09:35:19.841581 9822 ts_tablet_manager.cc:610] Registered 1 tablets
I20260809 09:35:19.841657 9822 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260809 09:35:19.842058 9892 tablet_bootstrap.cc:492] T 35f13e3276144674b6fb07b801a43225 P 2128f08f594542018da9bd8141d982c4: Bootstrap starting.
I20260809 09:35:19.852363 9822 rpc_server.cc:307] RPC server started. Bound to: 127.20.125.129:33947
I20260809 09:35:19.852470 10047 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.125.129:33947 every 8 connection(s)
I20260809 09:35:19.853367 9822 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/minicluster-data/ts-0/data/info.pb
I20260809 09:35:19.858536 20982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu as pid 9822
I20260809 09:35:19.859050 20982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
/tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.125.130:46209
--local_ip_for_outbound_sockets=127.20.125.130
--tserver_master_addrs=127.20.125.190:36897
--webserver_port=34923
--webserver_interface=127.20.125.130
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.130.keytab
--principal=kudu/127.20.125.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.125.148:40667
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 09:35:19.862041 9892 log.cc:824] T 35f13e3276144674b6fb07b801a43225 P 2128f08f594542018da9bd8141d982c4: Log is configured to *not* fsync() on all Append() calls
Aug 09 09:35:19 dist-test-slave-7kzw krb5kdc[8892](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268119, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.129@KRBTEST.COM for kudu/127.20.125.190@KRBTEST.COM
I20260809 09:35:19.871572 10051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:35:19.855514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.129:45893 (local address 127.20.125.190:36897)
0809 09:35:19.856065 (+ 551us) server_negotiation.cc:206] Beginning negotiation
0809 09:35:19.856086 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0809 09:35:19.856135 (+ 49us) server_negotiation.cc:407] Connection header received
0809 09:35:19.857406 (+ 1271us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:35:19.857422 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:35:19.857831 (+ 409us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:35:19.858257 (+ 426us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:35:19.859861 (+ 1604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:35:19.861326 (+ 1465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:35:19.863013 (+ 1687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:35:19.863525 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:35:19.863722 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:35:19.866342 (+ 2620us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:35:19.866347 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:35:19.866360 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:35:19.866389 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:35:19.868223 (+ 1834us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:35:19.868792 (+ 569us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:35:19.868798 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:35:19.868802 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:35:19.868926 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:35:19.869303 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:35:19.869310 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:35:19.869312 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:35:19.869655 (+ 343us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:35:19.869852 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0809 09:35:19.870400 (+ 548us) server_negotiation.cc:299] Negotiation successful
0809 09:35:19.870696 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":408,"thread_start_us":143,"threads_started":1}
I20260809 09:35:19.871587 10050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:35:19.855453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:36897 (local address 127.20.125.129:45893)
0809 09:35:19.855941 (+ 488us) negotiation.cc:107] Waiting for socket to connect
0809 09:35:19.855996 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0809 09:35:19.856946 (+ 950us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:35:19.858320 (+ 1374us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:35:19.858329 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:35:19.858954 (+ 625us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:35:19.859638 (+ 684us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:35:19.859662 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:35:19.861514 (+ 1852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:35:19.861520 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:35:19.862879 (+ 1359us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:35:19.862891 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:35:19.863683 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:35:19.863686 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:35:19.863886 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:35:19.864526 (+ 640us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 09:35:19.864544 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:35:19.866122 (+ 1578us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 09:35:19.868367 (+ 2245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:35:19.868373 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:35:19.868386 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:35:19.868659 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:35:19.869031 (+ 372us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:35:19.869035 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:35:19.869037 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:35:19.869184 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:35:19.869820 (+ 636us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 09:35:19.869826 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 09:35:19.870070 (+ 244us) client_negotiation.cc:769] Sending connection context
0809 09:35:19.870276 (+ 206us) client_negotiation.cc:240] Negotiation successful
0809 09:35:19.870664 (+ 388us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":282,"thread_start_us":145,"threads_started":1}
I20260809 09:35:19.872211 9892 tablet_bootstrap.cc:492] T 35f13e3276144674b6fb07b801a43225 P 2128f08f594542018da9bd8141d982c4: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 09:35:19.872725 9892 tablet_bootstrap.cc:492] T 35f13e3276144674b6fb07b801a43225 P 2128f08f594542018da9bd8141d982c4: Bootstrap complete.
I20260809 09:35:19.874321 9892 ts_tablet_manager.cc:1397] T 35f13e3276144674b6fb07b801a43225 P 2128f08f594542018da9bd8141d982c4: Time spent bootstrapping tablet: real 0.033s user 0.021s sys 0.008s
I20260809 09:35:19.875779 10048 heartbeater.cc:347] Connected to a master server at 127.20.125.190:36897
I20260809 09:35:19.876019 10048 heartbeater.cc:464] Registering TS with master...
I20260809 09:35:19.876514 10048 heartbeater.cc:511] Master 127.20.125.190:36897 requested a full tablet report, sending...
I20260809 09:35:19.877345 9892 raft_consensus.cc:348] T 35f13e3276144674b6fb07b801a43225 P 2128f08f594542018da9bd8141d982c4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26f94d9587f546bebd8f7351a6c30140" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 45239 } } peers { permanent_uuid: "c0b76307d73b4cfdba69b55655d1b26d" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 46209 } } peers { permanent_uuid: "2128f08f594542018da9bd8141d982c4" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 33947 } }
I20260809 09:35:19.877877 9892 raft_consensus.cc:729] T 35f13e3276144674b6fb07b801a43225 P 2128f08f594542018da9bd8141d982c4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2128f08f594542018da9bd8141d982c4, State: Initialized, Role: FOLLOWER
I20260809 09:35:19.878352 9892 consensus_queue.cc:260] T 35f13e3276144674b6fb07b801a43225 P 2128f08f594542018da9bd8141d982c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26f94d9587f546bebd8f7351a6c30140" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 45239 } } peers { permanent_uuid: "c0b76307d73b4cfdba69b55655d1b26d" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 46209 } } peers { permanent_uuid: "2128f08f594542018da9bd8141d982c4" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 33947 } }
I20260809 09:35:19.879114 9769 ts_manager.cc:194] Registered new tserver with Master: 2128f08f594542018da9bd8141d982c4 (127.20.125.129:33947)
I20260809 09:35:19.879441 9892 ts_tablet_manager.cc:1428] T 35f13e3276144674b6fb07b801a43225 P 2128f08f594542018da9bd8141d982c4: Time spent starting tablet: real 0.005s user 0.005s sys 0.003s
I20260809 09:35:19.882674 9769 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.125.129@KRBTEST.COM'} at 127.20.125.129:45893
I20260809 09:35:19.885298 10048 heartbeater.cc:503] Master 127.20.125.190:36897 was elected leader, sending a full tablet report...
Aug 09 09:35:19 dist-test-slave-7kzw krb5kdc[8892](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268119, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.190@KRBTEST.COM for kudu/127.20.125.129@KRBTEST.COM
I20260809 09:35:19.893692 10057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:35:19.881963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.129:33947 (local address 127.0.0.1:55762)
0809 09:35:19.882611 (+ 648us) negotiation.cc:107] Waiting for socket to connect
0809 09:35:19.882664 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0809 09:35:19.882810 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:35:19.883348 (+ 538us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:35:19.883354 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:35:19.883797 (+ 443us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:35:19.883929 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:35:19.883938 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:35:19.885234 (+ 1296us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:35:19.885238 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:35:19.886206 (+ 968us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:35:19.886218 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:35:19.887222 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:35:19.887226 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:35:19.887363 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:35:19.888239 (+ 876us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 09:35:19.888251 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:35:19.889782 (+ 1531us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 09:35:19.891831 (+ 2049us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:35:19.891843 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:35:19.891849 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:35:19.892087 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:35:19.892414 (+ 327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:35:19.892418 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:35:19.892420 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:35:19.892515 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:35:19.892982 (+ 467us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 09:35:19.892988 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 09:35:19.893146 (+ 158us) client_negotiation.cc:769] Sending connection context
0809 09:35:19.893299 (+ 153us) client_negotiation.cc:240] Negotiation successful
0809 09:35:19.893499 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":429,"thread_start_us":106,"threads_started":1}
I20260809 09:35:19.893697 10058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:35:19.882085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55762 (local address 127.20.125.129:33947)
0809 09:35:19.882708 (+ 623us) server_negotiation.cc:206] Beginning negotiation
0809 09:35:19.882723 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 09:35:19.882749 (+ 26us) server_negotiation.cc:407] Connection header received
0809 09:35:19.882974 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:35:19.882979 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:35:19.883176 (+ 197us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:35:19.883324 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:35:19.884065 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:35:19.885093 (+ 1028us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:35:19.886341 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:35:19.886915 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:35:19.887216 (+ 301us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:35:19.889933 (+ 2717us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:35:19.889943 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:35:19.889957 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:35:19.889988 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:35:19.891712 (+ 1724us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:35:19.892227 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:35:19.892233 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:35:19.892239 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:35:19.892318 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:35:19.892610 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:35:19.892614 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:35:19.892616 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:35:19.892862 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:35:19.893009 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0809 09:35:19.893382 (+ 373us) server_negotiation.cc:299] Negotiation successful
0809 09:35:19.893544 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":526,"thread_start_us":106,"threads_started":1}
I20260809 09:35:19.895347 9955 tablet_service.cc:1514] Processing DeleteTablet for tablet 35f13e3276144674b6fb07b801a43225 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 09:35:19 UTC) from {username='kudu', principal='kudu/127.20.125.190@KRBTEST.COM'} at 127.0.0.1:55762
I20260809 09:35:19.896204 10059 tablet_replica.cc:330] stopping tablet replica
I20260809 09:35:19.896524 10059 raft_consensus.cc:2229] T 35f13e3276144674b6fb07b801a43225 P 2128f08f594542018da9bd8141d982c4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 09:35:19.896786 10059 raft_consensus.cc:2258] T 35f13e3276144674b6fb07b801a43225 P 2128f08f594542018da9bd8141d982c4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 09:35:19.897797 10059 ts_tablet_manager.cc:1905] T 35f13e3276144674b6fb07b801a43225 P 2128f08f594542018da9bd8141d982c4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 09:35:19.901379 10059 ts_tablet_manager.cc:1918] T 35f13e3276144674b6fb07b801a43225 P 2128f08f594542018da9bd8141d982c4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 09:35:19.901467 10059 log.cc:1196] T 35f13e3276144674b6fb07b801a43225 P 2128f08f594542018da9bd8141d982c4: Deleting WAL directory at /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/minicluster-data/ts-0/wal/wals/35f13e3276144674b6fb07b801a43225
I20260809 09:35:19.901767 10059 ts_tablet_manager.cc:1939] T 35f13e3276144674b6fb07b801a43225 P 2128f08f594542018da9bd8141d982c4: Deleting consensus metadata
I20260809 09:35:19.903573 9748 catalog_manager.cc:4841] TS 2128f08f594542018da9bd8141d982c4 (127.20.125.129:33947): tablet 35f13e3276144674b6fb07b801a43225 (table test-table [id=734c86a9788f423083b7c0b8f72384a3]) successfully deleted
W20260809 09:35:19.984648 10053 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 09:35:19.984977 10053 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 09:35:19.985059 10053 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 09:35:19.989521 10053 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 09:35:19.989614 10053 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 09:35:19.989707 10053 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.125.130
I20260809 09:35:19.995050 10053 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.125.148:40667
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.130.keytab
--principal=kudu/127.20.125.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.125.130:46209
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.125.130
--webserver_port=34923
--tserver_master_addrs=127.20.125.190:36897
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10053
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.125.130
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 08:38:55 UTC on ccycloud.quasar-njjleu.root.comops.site
build id 4575935
I20260809 09:35:19.996186 10053 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 09:35:19.997362 10053 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 09:35:20.007587 10073 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:35:20.007668 10070 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:35:20.007604 10069 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:35:20.008145 10053 server_base.cc:1034] running on GCE node
I20260809 09:35:20.008662 10053 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 09:35:20.009410 10053 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 09:35:20.010627 10053 hybrid_clock.cc:648] HybridClock initialized: now 1786268120010616 us; error 56 us; skew 500 ppm
Aug 09 09:35:20 dist-test-slave-7kzw krb5kdc[8892](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268120, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 09:35:20.013978 10053 init.cc:377] Logged in from keytab as kudu/127.20.125.130@KRBTEST.COM (short username kudu)
I20260809 09:35:20.015018 10053 webserver.cc:460] Webserver started at http://127.20.125.130:34923/ using document root <none> and password file <none>
I20260809 09:35:20.015668 10053 fs_manager.cc:359] Metadata directory not provided
I20260809 09:35:20.015797 10053 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 09:35:20.020148 10053 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260809 09:35:20.022568 10086 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 09:35:20.024106 10053 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 09:35:20.024291 10053 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/minicluster-data/ts-1/data,/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/minicluster-data/ts-1/wal
uuid: "c0b76307d73b4cfdba69b55655d1b26d"
format_stamp: "Formatted at 2026-08-09 09:35:18 on dist-test-slave-7kzw"
server_key: "50584686f8518c032d7ff83ee5686320"
server_key_iv: "ce09d747ffff61d08f59d38ae4903162"
server_key_version: "encryptionkey@0"
I20260809 09:35:20.024966 10053 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 09:35:20.038044 10053 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 09:35:20.041664 10053 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 09:35:20.041887 10053 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 09:35:20.042600 10053 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 09:35:20.043481 10053 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 09:35:20.043557 10053 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 09:35:20.043627 10053 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 09:35:20.043689 10053 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 09:35:20.053426 10053 rpc_server.cc:307] RPC server started. Bound to: 127.20.125.130:46209
I20260809 09:35:20.053488 10252 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.125.130:46209 every 8 connection(s)
I20260809 09:35:20.054593 10053 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/minicluster-data/ts-1/data/info.pb
I20260809 09:35:20.058375 20982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu as pid 10053
I20260809 09:35:20.058918 20982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
/tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.125.131:45239
--local_ip_for_outbound_sockets=127.20.125.131
--tserver_master_addrs=127.20.125.190:36897
--webserver_port=41433
--webserver_interface=127.20.125.131
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.131.keytab
--principal=kudu/127.20.125.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.125.148:40667
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 09:35:20 dist-test-slave-7kzw krb5kdc[8892](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268120, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.130@KRBTEST.COM for kudu/127.20.125.190@KRBTEST.COM
I20260809 09:35:20.071425 10051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:35:20.056639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.130:34529 (local address 127.20.125.190:36897)
0809 09:35:20.056796 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0809 09:35:20.056800 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 09:35:20.057704 (+ 904us) server_negotiation.cc:407] Connection header received
0809 09:35:20.059301 (+ 1597us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:35:20.059304 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:35:20.059367 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:35:20.059490 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:35:20.060729 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:35:20.061285 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:35:20.062575 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:35:20.062845 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:35:20.062948 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:35:20.066901 (+ 3953us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:35:20.066904 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:35:20.066907 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:35:20.066936 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:35:20.068668 (+ 1732us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:35:20.069429 (+ 761us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:35:20.069433 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:35:20.069434 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:35:20.069524 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:35:20.069994 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:35:20.069997 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:35:20.069998 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:35:20.070192 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:35:20.070284 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0809 09:35:20.071139 (+ 855us) server_negotiation.cc:299] Negotiation successful
0809 09:35:20.071282 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260809 09:35:20.072921 10257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:35:20.057049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:36897 (local address 127.20.125.130:34529)
0809 09:35:20.057537 (+ 488us) negotiation.cc:107] Waiting for socket to connect
0809 09:35:20.057581 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 09:35:20.059014 (+ 1433us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:35:20.059679 (+ 665us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:35:20.059686 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:35:20.060059 (+ 373us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:35:20.060548 (+ 489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:35:20.060562 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:35:20.061432 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:35:20.061436 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:35:20.062375 (+ 939us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:35:20.062390 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:35:20.063128 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:35:20.063137 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:35:20.063443 (+ 306us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:35:20.064273 (+ 830us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 09:35:20.064306 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:35:20.066674 (+ 2368us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 09:35:20.068812 (+ 2138us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:35:20.068821 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:35:20.068844 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:35:20.069294 (+ 450us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:35:20.069633 (+ 339us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:35:20.069640 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:35:20.069645 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:35:20.069877 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:35:20.070317 (+ 440us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 09:35:20.070326 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 09:35:20.070827 (+ 501us) client_negotiation.cc:769] Sending connection context
0809 09:35:20.071106 (+ 279us) client_negotiation.cc:240] Negotiation successful
0809 09:35:20.071574 (+ 468us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":299,"thread_start_us":131,"threads_started":1}
I20260809 09:35:20.074307 10254 heartbeater.cc:347] Connected to a master server at 127.20.125.190:36897
I20260809 09:35:20.074622 10254 heartbeater.cc:464] Registering TS with master...
I20260809 09:35:20.075237 10254 heartbeater.cc:511] Master 127.20.125.190:36897 requested a full tablet report, sending...
I20260809 09:35:20.076453 9769 ts_manager.cc:194] Registered new tserver with Master: c0b76307d73b4cfdba69b55655d1b26d (127.20.125.130:46209)
I20260809 09:35:20.077095 9769 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.125.130@KRBTEST.COM'} at 127.20.125.130:34529
W20260809 09:35:20.188446 10258 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 09:35:20.188781 10258 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 09:35:20.188869 10258 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 09:35:20.192958 10258 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 09:35:20.193048 10258 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 09:35:20.193131 10258 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.125.131
I20260809 09:35:20.198312 10258 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.125.148:40667
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.131.keytab
--principal=kudu/127.20.125.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.125.131:45239
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.125.131
--webserver_port=41433
--tserver_master_addrs=127.20.125.190:36897
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10258
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.125.131
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 08:38:55 UTC on ccycloud.quasar-njjleu.root.comops.site
build id 4575935
I20260809 09:35:20.199402 10258 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 09:35:20.200475 10258 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 09:35:20.211464 10258 server_base.cc:1034] running on GCE node
W20260809 09:35:20.211421 10267 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:35:20.211426 10266 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:35:20.211421 10271 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:35:20.212216 10258 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 09:35:20.212858 10258 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 09:35:20.214051 10258 hybrid_clock.cc:648] HybridClock initialized: now 1786268120214039 us; error 26 us; skew 500 ppm
Aug 09 09:35:20 dist-test-slave-7kzw krb5kdc[8892](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268120, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 09:35:20.217010 10258 init.cc:377] Logged in from keytab as kudu/127.20.125.131@KRBTEST.COM (short username kudu)
I20260809 09:35:20.217895 10258 webserver.cc:460] Webserver started at http://127.20.125.131:41433/ using document root <none> and password file <none>
I20260809 09:35:20.218547 10258 fs_manager.cc:359] Metadata directory not provided
I20260809 09:35:20.218657 10258 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 09:35:20.223061 10258 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260809 09:35:20.225059 10278 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 09:35:20.226351 10258 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 09:35:20.226512 10258 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/minicluster-data/ts-2/data,/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/minicluster-data/ts-2/wal
uuid: "26f94d9587f546bebd8f7351a6c30140"
format_stamp: "Formatted at 2026-08-09 09:35:19 on dist-test-slave-7kzw"
server_key: "1d155273c6f05f49880d6d2b74b34b6c"
server_key_iv: "e9ff8f13dc172def69b26f94e67bf0bc"
server_key_version: "encryptionkey@0"
I20260809 09:35:20.227058 10258 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 09:35:20.242846 10258 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 09:35:20.246248 10258 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 09:35:20.246413 10258 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 09:35:20.247061 10258 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 09:35:20.247947 10258 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 09:35:20.247995 10258 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 09:35:20.248066 10258 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 09:35:20.248099 10258 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 09:35:20.257153 10258 rpc_server.cc:307] RPC server started. Bound to: 127.20.125.131:45239
I20260809 09:35:20.257195 10448 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.125.131:45239 every 8 connection(s)
I20260809 09:35:20.258185 10258 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/minicluster-data/ts-2/data/info.pb
I20260809 09:35:20.258652 20982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu as pid 10258
I20260809 09:35:20.268285 10051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:35:20.259443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50282 (local address 127.20.125.190:36897)
0809 09:35:20.259607 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0809 09:35:20.259612 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 09:35:20.259879 (+ 267us) server_negotiation.cc:407] Connection header received
0809 09:35:20.259990 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:35:20.259996 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:35:20.260054 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:35:20.260143 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:35:20.260964 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:35:20.261689 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:35:20.262972 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:35:20.263282 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:35:20.263362 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:35:20.264999 (+ 1637us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:35:20.265003 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:35:20.265005 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:35:20.265033 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:35:20.266546 (+ 1513us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:35:20.266994 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:35:20.266998 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:35:20.266999 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:35:20.267089 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:35:20.267410 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:35:20.267413 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:35:20.267414 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:35:20.267571 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:35:20.267811 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0809 09:35:20.267989 (+ 178us) server_negotiation.cc:299] Negotiation successful
0809 09:35:20.268111 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
Aug 09 09:35:20 dist-test-slave-7kzw krb5kdc[8892](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268120, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.131@KRBTEST.COM for kudu/127.20.125.190@KRBTEST.COM
I20260809 09:35:20.273828 10453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:35:20.260153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.131:50071 (local address 127.20.125.190:36897)
0809 09:35:20.260357 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0809 09:35:20.260361 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 09:35:20.261278 (+ 917us) server_negotiation.cc:407] Connection header received
0809 09:35:20.262974 (+ 1696us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:35:20.262977 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:35:20.263027 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:35:20.263116 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:35:20.264717 (+ 1601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:35:20.265388 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:35:20.266796 (+ 1408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:35:20.267072 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:35:20.267191 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:35:20.270114 (+ 2923us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:35:20.270118 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:35:20.270120 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:35:20.270144 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:35:20.271747 (+ 1603us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:35:20.272275 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:35:20.272279 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:35:20.272280 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:35:20.272370 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:35:20.272722 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:35:20.272727 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:35:20.272728 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:35:20.272912 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:35:20.273020 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0809 09:35:20.273521 (+ 501us) server_negotiation.cc:299] Negotiation successful
0809 09:35:20.273643 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":77,"threads_started":1}
I20260809 09:35:20.274838 10454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:35:20.260656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:36897 (local address 127.20.125.131:50071)
0809 09:35:20.261139 (+ 483us) negotiation.cc:107] Waiting for socket to connect
0809 09:35:20.261183 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 09:35:20.262756 (+ 1573us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:35:20.263282 (+ 526us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:35:20.263288 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:35:20.263830 (+ 542us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:35:20.264542 (+ 712us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:35:20.264566 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:35:20.265547 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:35:20.265556 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:35:20.266663 (+ 1107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:35:20.266675 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:35:20.267241 (+ 566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:35:20.267248 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:35:20.267475 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:35:20.268093 (+ 618us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 09:35:20.268117 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:35:20.269948 (+ 1831us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 09:35:20.271872 (+ 1924us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:35:20.271878 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:35:20.271895 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:35:20.272160 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:35:20.272464 (+ 304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:35:20.272468 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:35:20.272470 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:35:20.272618 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:35:20.273019 (+ 401us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 09:35:20.273026 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 09:35:20.273285 (+ 259us) client_negotiation.cc:769] Sending connection context
0809 09:35:20.273478 (+ 193us) client_negotiation.cc:240] Negotiation successful
0809 09:35:20.273802 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":286,"thread_start_us":101,"threads_started":1}
I20260809 09:35:20.278578 10449 heartbeater.cc:347] Connected to a master server at 127.20.125.190:36897
I20260809 09:35:20.278879 10449 heartbeater.cc:464] Registering TS with master...
I20260809 09:35:20.279578 10449 heartbeater.cc:511] Master 127.20.125.190:36897 requested a full tablet report, sending...
I20260809 09:35:20.280987 9769 ts_manager.cc:194] Registered new tserver with Master: 26f94d9587f546bebd8f7351a6c30140 (127.20.125.131:45239)
I20260809 09:35:20.281805 9769 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.125.131@KRBTEST.COM'} at 127.20.125.131:50071
I20260809 09:35:20.284701 20982 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 09:35:20.291041 10453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:35:20.285381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50292 (local address 127.20.125.190:36897)
0809 09:35:20.286219 (+ 838us) server_negotiation.cc:206] Beginning negotiation
0809 09:35:20.286224 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 09:35:20.286433 (+ 209us) server_negotiation.cc:407] Connection header received
0809 09:35:20.286671 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:35:20.286674 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:35:20.286804 (+ 130us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:35:20.286890 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 09:35:20.287328 (+ 438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:35:20.288067 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:35:20.289275 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:35:20.289686 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:35:20.289794 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:35:20.290194 (+ 400us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 09:35:20.290635 (+ 441us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 09:35:20.290732 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0809 09:35:20.290846 (+ 114us) server_negotiation.cc:299] Negotiation successful
0809 09:35:20.290901 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":744}
I20260809 09:35:20.291821 9769 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:50292:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 09:35:20.297215 9769 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 09:35:20.310310 9955 tablet_service.cc:1467] Processing CreateTablet for tablet 84b14f1c3fd64b719acbf700e2dbdbe2 (DEFAULT_TABLE table=test-table [id=c7c2ed2765cd424380b0d033ceb24e63]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 09:35:20.311084 9955 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 84b14f1c3fd64b719acbf700e2dbdbe2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 09:35:20.314155 9892 tablet_bootstrap.cc:492] T 84b14f1c3fd64b719acbf700e2dbdbe2 P 2128f08f594542018da9bd8141d982c4: Bootstrap starting.
I20260809 09:35:20.314455 10464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:35:20.308667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.131:45239 (local address 127.0.0.1:51514)
0809 09:35:20.309463 (+ 796us) negotiation.cc:107] Waiting for socket to connect
0809 09:35:20.309478 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 09:35:20.309593 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:35:20.310085 (+ 492us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:35:20.310091 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:35:20.310116 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 09:35:20.310251 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:35:20.310261 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:35:20.311670 (+ 1409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:35:20.311677 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:35:20.312798 (+ 1121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:35:20.312809 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:35:20.313968 (+ 1159us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:35:20.313971 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:35:20.314087 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:35:20.314140 (+ 53us) client_negotiation.cc:769] Sending connection context
0809 09:35:20.314233 (+ 93us) client_negotiation.cc:240] Negotiation successful
0809 09:35:20.314310 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":732,"thread_start_us":660,"threads_started":1}
I20260809 09:35:20.314570 10462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:35:20.306753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51514 (local address 127.20.125.131:45239)
0809 09:35:20.307123 (+ 370us) server_negotiation.cc:206] Beginning negotiation
0809 09:35:20.307134 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 09:35:20.309595 (+ 2461us) server_negotiation.cc:407] Connection header received
0809 09:35:20.309713 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:35:20.309719 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:35:20.309918 (+ 199us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:35:20.310083 (+ 165us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 09:35:20.310406 (+ 323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:35:20.311490 (+ 1084us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:35:20.312934 (+ 1444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:35:20.313838 (+ 904us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:35:20.313976 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:35:20.314118 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0809 09:35:20.314286 (+ 168us) server_negotiation.cc:299] Negotiation successful
0809 09:35:20.314388 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":254,"thread_start_us":122,"threads_started":1}
I20260809 09:35:20.315143 9892 tablet_bootstrap.cc:654] T 84b14f1c3fd64b719acbf700e2dbdbe2 P 2128f08f594542018da9bd8141d982c4: Neither blocks nor log segments found. Creating new log.
I20260809 09:35:20.317193 9892 tablet_bootstrap.cc:492] T 84b14f1c3fd64b719acbf700e2dbdbe2 P 2128f08f594542018da9bd8141d982c4: No bootstrap required, opened a new log
I20260809 09:35:20.317287 9892 ts_tablet_manager.cc:1397] T 84b14f1c3fd64b719acbf700e2dbdbe2 P 2128f08f594542018da9bd8141d982c4: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 09:35:20.317768 9892 raft_consensus.cc:348] T 84b14f1c3fd64b719acbf700e2dbdbe2 P 2128f08f594542018da9bd8141d982c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26f94d9587f546bebd8f7351a6c30140" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 45239 } } peers { permanent_uuid: "c0b76307d73b4cfdba69b55655d1b26d" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 46209 } } peers { permanent_uuid: "2128f08f594542018da9bd8141d982c4" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 33947 } }
I20260809 09:35:20.317909 9892 raft_consensus.cc:374] T 84b14f1c3fd64b719acbf700e2dbdbe2 P 2128f08f594542018da9bd8141d982c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 09:35:20.317947 9892 raft_consensus.cc:729] T 84b14f1c3fd64b719acbf700e2dbdbe2 P 2128f08f594542018da9bd8141d982c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2128f08f594542018da9bd8141d982c4, State: Initialized, Role: FOLLOWER
I20260809 09:35:20.318063 10057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:35:20.306417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.130:46209 (local address 127.0.0.1:44708)
0809 09:35:20.306816 (+ 399us) negotiation.cc:107] Waiting for socket to connect
0809 09:35:20.306832 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 09:35:20.306946 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:35:20.309083 (+ 2137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:35:20.309086 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:35:20.309107 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 09:35:20.309256 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:35:20.309264 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:35:20.315081 (+ 5817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:35:20.315087 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:35:20.316188 (+ 1101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:35:20.316199 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:35:20.317690 (+ 1491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:35:20.317694 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:35:20.317793 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:35:20.317818 (+ 25us) client_negotiation.cc:769] Sending connection context
0809 09:35:20.317870 (+ 52us) client_negotiation.cc:240] Negotiation successful
0809 09:35:20.317943 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":294}
I20260809 09:35:20.318107 9892 consensus_queue.cc:260] T 84b14f1c3fd64b719acbf700e2dbdbe2 P 2128f08f594542018da9bd8141d982c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26f94d9587f546bebd8f7351a6c30140" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 45239 } } peers { permanent_uuid: "c0b76307d73b4cfdba69b55655d1b26d" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 46209 } } peers { permanent_uuid: "2128f08f594542018da9bd8141d982c4" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 33947 } }
I20260809 09:35:20.318811 10463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:35:20.307188 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44708 (local address 127.20.125.130:46209)
0809 09:35:20.308574 (+ 1386us) server_negotiation.cc:206] Beginning negotiation
0809 09:35:20.308587 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 09:35:20.308610 (+ 23us) server_negotiation.cc:407] Connection header received
0809 09:35:20.308712 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:35:20.308716 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:35:20.308898 (+ 182us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:35:20.309095 (+ 197us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 09:35:20.309360 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:35:20.314941 (+ 5581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:35:20.316361 (+ 1420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:35:20.317541 (+ 1180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:35:20.318360 (+ 819us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:35:20.318519 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0809 09:35:20.318584 (+ 65us) server_negotiation.cc:299] Negotiation successful
0809 09:35:20.318671 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1278,"thread_start_us":1160,"threads_started":1}
I20260809 09:35:20.320271 9892 ts_tablet_manager.cc:1428] T 84b14f1c3fd64b719acbf700e2dbdbe2 P 2128f08f594542018da9bd8141d982c4: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260809 09:35:20.320456 10350 tablet_service.cc:1467] Processing CreateTablet for tablet 84b14f1c3fd64b719acbf700e2dbdbe2 (DEFAULT_TABLE table=test-table [id=c7c2ed2765cd424380b0d033ceb24e63]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 09:35:20.321580 10350 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 84b14f1c3fd64b719acbf700e2dbdbe2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 09:35:20.321521 10161 tablet_service.cc:1467] Processing CreateTablet for tablet 84b14f1c3fd64b719acbf700e2dbdbe2 (DEFAULT_TABLE table=test-table [id=c7c2ed2765cd424380b0d033ceb24e63]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 09:35:20.322536 10161 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 84b14f1c3fd64b719acbf700e2dbdbe2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 09:35:20.328858 10466 tablet_bootstrap.cc:492] T 84b14f1c3fd64b719acbf700e2dbdbe2 P 26f94d9587f546bebd8f7351a6c30140: Bootstrap starting.
I20260809 09:35:20.332649 10466 tablet_bootstrap.cc:654] T 84b14f1c3fd64b719acbf700e2dbdbe2 P 26f94d9587f546bebd8f7351a6c30140: Neither blocks nor log segments found. Creating new log.
I20260809 09:35:20.333519 10466 log.cc:824] T 84b14f1c3fd64b719acbf700e2dbdbe2 P 26f94d9587f546bebd8f7351a6c30140: Log is configured to *not* fsync() on all Append() calls
I20260809 09:35:20.334616 10467 tablet_bootstrap.cc:492] T 84b14f1c3fd64b719acbf700e2dbdbe2 P c0b76307d73b4cfdba69b55655d1b26d: Bootstrap starting.
I20260809 09:35:20.335947 10466 tablet_bootstrap.cc:492] T 84b14f1c3fd64b719acbf700e2dbdbe2 P 26f94d9587f546bebd8f7351a6c30140: No bootstrap required, opened a new log
I20260809 09:35:20.336169 10466 ts_tablet_manager.cc:1397] T 84b14f1c3fd64b719acbf700e2dbdbe2 P 26f94d9587f546bebd8f7351a6c30140: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260809 09:35:20.337430 10467 tablet_bootstrap.cc:654] T 84b14f1c3fd64b719acbf700e2dbdbe2 P c0b76307d73b4cfdba69b55655d1b26d: Neither blocks nor log segments found. Creating new log.
I20260809 09:35:20.338302 10467 log.cc:824] T 84b14f1c3fd64b719acbf700e2dbdbe2 P c0b76307d73b4cfdba69b55655d1b26d: Log is configured to *not* fsync() on all Append() calls
I20260809 09:35:20.340565 10467 tablet_bootstrap.cc:492] T 84b14f1c3fd64b719acbf700e2dbdbe2 P c0b76307d73b4cfdba69b55655d1b26d: No bootstrap required, opened a new log
I20260809 09:35:20.340771 10467 ts_tablet_manager.cc:1397] T 84b14f1c3fd64b719acbf700e2dbdbe2 P c0b76307d73b4cfdba69b55655d1b26d: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.002s
I20260809 09:35:20.341066 10466 raft_consensus.cc:348] T 84b14f1c3fd64b719acbf700e2dbdbe2 P 26f94d9587f546bebd8f7351a6c30140 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26f94d9587f546bebd8f7351a6c30140" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 45239 } } peers { permanent_uuid: "c0b76307d73b4cfdba69b55655d1b26d" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 46209 } } peers { permanent_uuid: "2128f08f594542018da9bd8141d982c4" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 33947 } }
I20260809 09:35:20.341394 10466 raft_consensus.cc:374] T 84b14f1c3fd64b719acbf700e2dbdbe2 P 26f94d9587f546bebd8f7351a6c30140 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 09:35:20.341465 10466 raft_consensus.cc:729] T 84b14f1c3fd64b719acbf700e2dbdbe2 P 26f94d9587f546bebd8f7351a6c30140 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 26f94d9587f546bebd8f7351a6c30140, State: Initialized, Role: FOLLOWER
I20260809 09:35:20.341971 10466 consensus_queue.cc:260] T 84b14f1c3fd64b719acbf700e2dbdbe2 P 26f94d9587f546bebd8f7351a6c30140 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26f94d9587f546bebd8f7351a6c30140" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 45239 } } peers { permanent_uuid: "c0b76307d73b4cfdba69b55655d1b26d" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 46209 } } peers { permanent_uuid: "2128f08f594542018da9bd8141d982c4" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 33947 } }
I20260809 09:35:20.342761 10449 heartbeater.cc:503] Master 127.20.125.190:36897 was elected leader, sending a full tablet report...
I20260809 09:35:20.343259 10466 ts_tablet_manager.cc:1428] T 84b14f1c3fd64b719acbf700e2dbdbe2 P 26f94d9587f546bebd8f7351a6c30140: Time spent starting tablet: real 0.007s user 0.008s sys 0.000s
I20260809 09:35:20.344733 10467 raft_consensus.cc:348] T 84b14f1c3fd64b719acbf700e2dbdbe2 P c0b76307d73b4cfdba69b55655d1b26d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26f94d9587f546bebd8f7351a6c30140" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 45239 } } peers { permanent_uuid: "c0b76307d73b4cfdba69b55655d1b26d" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 46209 } } peers { permanent_uuid: "2128f08f594542018da9bd8141d982c4" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 33947 } }
I20260809 09:35:20.344990 10467 raft_consensus.cc:374] T 84b14f1c3fd64b719acbf700e2dbdbe2 P c0b76307d73b4cfdba69b55655d1b26d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 09:35:20.345049 10467 raft_consensus.cc:729] T 84b14f1c3fd64b719acbf700e2dbdbe2 P c0b76307d73b4cfdba69b55655d1b26d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c0b76307d73b4cfdba69b55655d1b26d, State: Initialized, Role: FOLLOWER
I20260809 09:35:20.345484 10467 consensus_queue.cc:260] T 84b14f1c3fd64b719acbf700e2dbdbe2 P c0b76307d73b4cfdba69b55655d1b26d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26f94d9587f546bebd8f7351a6c30140" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 45239 } } peers { permanent_uuid: "c0b76307d73b4cfdba69b55655d1b26d" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 46209 } } peers { permanent_uuid: "2128f08f594542018da9bd8141d982c4" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 33947 } }
I20260809 09:35:20.346122 10254 heartbeater.cc:503] Master 127.20.125.190:36897 was elected leader, sending a full tablet report...
I20260809 09:35:20.346550 10467 ts_tablet_manager.cc:1428] T 84b14f1c3fd64b719acbf700e2dbdbe2 P c0b76307d73b4cfdba69b55655d1b26d: Time spent starting tablet: real 0.006s user 0.006s sys 0.001s
W20260809 09:35:20.354683 10049 tablet.cc:2369] T 84b14f1c3fd64b719acbf700e2dbdbe2 P 2128f08f594542018da9bd8141d982c4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 09:35:20.395996 10470 raft_consensus.cc:482] T 84b14f1c3fd64b719acbf700e2dbdbe2 P 26f94d9587f546bebd8f7351a6c30140 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 09:35:20.396261 10470 raft_consensus.cc:504] T 84b14f1c3fd64b719acbf700e2dbdbe2 P 26f94d9587f546bebd8f7351a6c30140 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26f94d9587f546bebd8f7351a6c30140" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 45239 } } peers { permanent_uuid: "c0b76307d73b4cfdba69b55655d1b26d" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 46209 } } peers { permanent_uuid: "2128f08f594542018da9bd8141d982c4" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 33947 } }
I20260809 09:35:20.397439 10470 leader_election.cc:288] T 84b14f1c3fd64b719acbf700e2dbdbe2 P 26f94d9587f546bebd8f7351a6c30140 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c0b76307d73b4cfdba69b55655d1b26d (127.20.125.130:46209), 2128f08f594542018da9bd8141d982c4 (127.20.125.129:33947)
I20260809 09:35:20.401387 10473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:35:20.397820 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.131:53667 (local address 127.20.125.129:33947)
0809 09:35:20.398099 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0809 09:35:20.398104 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 09:35:20.398117 (+ 13us) server_negotiation.cc:407] Connection header received
0809 09:35:20.398172 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:35:20.398175 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:35:20.398232 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:35:20.398347 (+ 115us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 09:35:20.398638 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:35:20.399192 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:35:20.400239 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:35:20.400697 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:35:20.400836 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:35:20.401001 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0809 09:35:20.401166 (+ 165us) server_negotiation.cc:299] Negotiation successful
0809 09:35:20.401238 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":193,"thread_start_us":89,"threads_started":1}
I20260809 09:35:20.401389 10463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:35:20.397815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.131:51091 (local address 127.20.125.130:46209)
0809 09:35:20.397964 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0809 09:35:20.397971 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 09:35:20.398251 (+ 280us) server_negotiation.cc:407] Connection header received
0809 09:35:20.398414 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:35:20.398416 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:35:20.398507 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:35:20.398601 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 09:35:20.398862 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:35:20.399340 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:35:20.400336 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:35:20.400722 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:35:20.400847 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:35:20.400903 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0809 09:35:20.401197 (+ 294us) server_negotiation.cc:299] Negotiation successful
0809 09:35:20.401268 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260809 09:35:20.401386 10454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:35:20.397676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.129:33947 (local address 127.20.125.131:53667)
0809 09:35:20.397906 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0809 09:35:20.397923 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 09:35:20.398014 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:35:20.398357 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:35:20.398361 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:35:20.398379 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 09:35:20.398519 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:35:20.398526 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:35:20.399306 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:35:20.399311 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:35:20.400106 (+ 795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:35:20.400115 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:35:20.400824 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:35:20.400827 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:35:20.401028 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:35:20.401052 (+ 24us) client_negotiation.cc:769] Sending connection context
0809 09:35:20.401148 (+ 96us) client_negotiation.cc:240] Negotiation successful
0809 09:35:20.401237 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"mutex_wait_us":53}
I20260809 09:35:20.401409 10472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:35:20.397683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.130:46209 (local address 127.20.125.131:51091)
0809 09:35:20.398134 (+ 451us) negotiation.cc:107] Waiting for socket to connect
0809 09:35:20.398146 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 09:35:20.398285 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:35:20.398609 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:35:20.398612 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:35:20.398628 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 09:35:20.398753 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:35:20.398759 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:35:20.399452 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:35:20.399461 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:35:20.400214 (+ 753us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:35:20.400222 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:35:20.400846 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:35:20.400850 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:35:20.400998 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:35:20.401079 (+ 81us) client_negotiation.cc:769] Sending connection context
0809 09:35:20.401172 (+ 93us) client_negotiation.cc:240] Negotiation successful
0809 09:35:20.401245 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":396,"mutex_wait_us":86,"thread_start_us":121,"threads_started":1}
I20260809 09:35:20.401990 9981 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84b14f1c3fd64b719acbf700e2dbdbe2" candidate_uuid: "26f94d9587f546bebd8f7351a6c30140" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2128f08f594542018da9bd8141d982c4" is_pre_election: true
I20260809 09:35:20.402256 10185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84b14f1c3fd64b719acbf700e2dbdbe2" candidate_uuid: "26f94d9587f546bebd8f7351a6c30140" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c0b76307d73b4cfdba69b55655d1b26d" is_pre_election: true
I20260809 09:35:20.402295 9981 raft_consensus.cc:2454] T 84b14f1c3fd64b719acbf700e2dbdbe2 P 2128f08f594542018da9bd8141d982c4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 26f94d9587f546bebd8f7351a6c30140 in term 0.
I20260809 09:35:20.402580 10185 raft_consensus.cc:2454] T 84b14f1c3fd64b719acbf700e2dbdbe2 P c0b76307d73b4cfdba69b55655d1b26d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 26f94d9587f546bebd8f7351a6c30140 in term 0.
I20260809 09:35:20.402815 10280 leader_election.cc:302] T 84b14f1c3fd64b719acbf700e2dbdbe2 P 26f94d9587f546bebd8f7351a6c30140 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2128f08f594542018da9bd8141d982c4, 26f94d9587f546bebd8f7351a6c30140; no voters:
I20260809 09:35:20.403110 10470 raft_consensus.cc:2781] T 84b14f1c3fd64b719acbf700e2dbdbe2 P 26f94d9587f546bebd8f7351a6c30140 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 09:35:20.403214 10470 raft_consensus.cc:482] T 84b14f1c3fd64b719acbf700e2dbdbe2 P 26f94d9587f546bebd8f7351a6c30140 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 09:35:20.403273 10470 raft_consensus.cc:3037] T 84b14f1c3fd64b719acbf700e2dbdbe2 P 26f94d9587f546bebd8f7351a6c30140 [term 0 FOLLOWER]: Advancing to term 1
I20260809 09:35:20.404398 10470 raft_consensus.cc:504] T 84b14f1c3fd64b719acbf700e2dbdbe2 P 26f94d9587f546bebd8f7351a6c30140 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26f94d9587f546bebd8f7351a6c30140" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 45239 } } peers { permanent_uuid: "c0b76307d73b4cfdba69b55655d1b26d" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 46209 } } peers { permanent_uuid: "2128f08f594542018da9bd8141d982c4" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 33947 } }
I20260809 09:35:20.404728 10470 leader_election.cc:288] T 84b14f1c3fd64b719acbf700e2dbdbe2 P 26f94d9587f546bebd8f7351a6c30140 [CANDIDATE]: Term 1 election: Requested vote from peers c0b76307d73b4cfdba69b55655d1b26d (127.20.125.130:46209), 2128f08f594542018da9bd8141d982c4 (127.20.125.129:33947)
I20260809 09:35:20.405129 10185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84b14f1c3fd64b719acbf700e2dbdbe2" candidate_uuid: "26f94d9587f546bebd8f7351a6c30140" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c0b76307d73b4cfdba69b55655d1b26d"
I20260809 09:35:20.405129 9981 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84b14f1c3fd64b719acbf700e2dbdbe2" candidate_uuid: "26f94d9587f546bebd8f7351a6c30140" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2128f08f594542018da9bd8141d982c4"
I20260809 09:35:20.405268 10185 raft_consensus.cc:3037] T 84b14f1c3fd64b719acbf700e2dbdbe2 P c0b76307d73b4cfdba69b55655d1b26d [term 0 FOLLOWER]: Advancing to term 1
I20260809 09:35:20.405273 9981 raft_consensus.cc:3037] T 84b14f1c3fd64b719acbf700e2dbdbe2 P 2128f08f594542018da9bd8141d982c4 [term 0 FOLLOWER]: Advancing to term 1
I20260809 09:35:20.406594 10185 raft_consensus.cc:2454] T 84b14f1c3fd64b719acbf700e2dbdbe2 P c0b76307d73b4cfdba69b55655d1b26d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 26f94d9587f546bebd8f7351a6c30140 in term 1.
I20260809 09:35:20.406602 9981 raft_consensus.cc:2454] T 84b14f1c3fd64b719acbf700e2dbdbe2 P 2128f08f594542018da9bd8141d982c4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 26f94d9587f546bebd8f7351a6c30140 in term 1.
I20260809 09:35:20.406908 10281 leader_election.cc:302] T 84b14f1c3fd64b719acbf700e2dbdbe2 P 26f94d9587f546bebd8f7351a6c30140 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 26f94d9587f546bebd8f7351a6c30140, c0b76307d73b4cfdba69b55655d1b26d; no voters:
I20260809 09:35:20.407079 10470 raft_consensus.cc:2781] T 84b14f1c3fd64b719acbf700e2dbdbe2 P 26f94d9587f546bebd8f7351a6c30140 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 09:35:20.407289 10470 raft_consensus.cc:686] T 84b14f1c3fd64b719acbf700e2dbdbe2 P 26f94d9587f546bebd8f7351a6c30140 [term 1 LEADER]: Becoming Leader. State: Replica: 26f94d9587f546bebd8f7351a6c30140, State: Running, Role: LEADER
I20260809 09:35:20.407631 10470 consensus_queue.cc:237] T 84b14f1c3fd64b719acbf700e2dbdbe2 P 26f94d9587f546bebd8f7351a6c30140 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26f94d9587f546bebd8f7351a6c30140" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 45239 } } peers { permanent_uuid: "c0b76307d73b4cfdba69b55655d1b26d" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 46209 } } peers { permanent_uuid: "2128f08f594542018da9bd8141d982c4" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 33947 } }
I20260809 09:35:20.410787 9769 catalog_manager.cc:5495] T 84b14f1c3fd64b719acbf700e2dbdbe2 P 26f94d9587f546bebd8f7351a6c30140 reported cstate change: term changed from 0 to 1, leader changed from <none> to 26f94d9587f546bebd8f7351a6c30140 (127.20.125.131). New cstate: current_term: 1 leader_uuid: "26f94d9587f546bebd8f7351a6c30140" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26f94d9587f546bebd8f7351a6c30140" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 45239 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c0b76307d73b4cfdba69b55655d1b26d" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 46209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2128f08f594542018da9bd8141d982c4" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 33947 } health_report { overall_health: UNKNOWN } } }
I20260809 09:35:20.423611 10462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:35:20.420487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51516 (local address 127.20.125.131:45239)
0809 09:35:20.420604 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0809 09:35:20.420608 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 09:35:20.420620 (+ 12us) server_negotiation.cc:407] Connection header received
0809 09:35:20.420748 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:35:20.420750 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:35:20.420800 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:35:20.420885 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 09:35:20.421074 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:35:20.421551 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:35:20.422345 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:35:20.422604 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:35:20.422704 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:35:20.422940 (+ 236us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 09:35:20.423317 (+ 377us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 09:35:20.423404 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0809 09:35:20.423465 (+ 61us) server_negotiation.cc:299] Negotiation successful
0809 09:35:20.423510 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260809 09:35:20.429466 9981 raft_consensus.cc:1261] T 84b14f1c3fd64b719acbf700e2dbdbe2 P 2128f08f594542018da9bd8141d982c4 [term 1 FOLLOWER]: Refusing update from remote peer 26f94d9587f546bebd8f7351a6c30140: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 09:35:20.429468 10185 raft_consensus.cc:1261] T 84b14f1c3fd64b719acbf700e2dbdbe2 P c0b76307d73b4cfdba69b55655d1b26d [term 1 FOLLOWER]: Refusing update from remote peer 26f94d9587f546bebd8f7351a6c30140: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 09:35:20.430227 10470 consensus_queue.cc:1035] T 84b14f1c3fd64b719acbf700e2dbdbe2 P 26f94d9587f546bebd8f7351a6c30140 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2128f08f594542018da9bd8141d982c4" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 33947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 09:35:20.430498 10476 consensus_queue.cc:1035] T 84b14f1c3fd64b719acbf700e2dbdbe2 P 26f94d9587f546bebd8f7351a6c30140 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c0b76307d73b4cfdba69b55655d1b26d" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 46209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 09:35:20.437392 10483 mvcc.cc:205] Tried to move back new op lower bound from 7316554221271822336 to 7316554221192151040. Current Snapshot: MvccSnapshot[applied={T|T < 7316554221271822336}]
I20260809 09:35:20.439637 10484 mvcc.cc:205] Tried to move back new op lower bound from 7316554221271822336 to 7316554221192151040. Current Snapshot: MvccSnapshot[applied={T|T < 7316554221271822336}]
I20260809 09:35:20.444754 9769 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:50292:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 09:35:20.444929 9769 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:50292:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 09:35:20.447434 9769 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7a385d41008d4b7aa6e916cf8f8ab796: Sending DeleteTablet for 3 replicas of tablet 84b14f1c3fd64b719acbf700e2dbdbe2
I20260809 09:35:20.448231 9955 tablet_service.cc:1514] Processing DeleteTablet for tablet 84b14f1c3fd64b719acbf700e2dbdbe2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 09:35:20 UTC) from {username='kudu', principal='kudu/127.20.125.190@KRBTEST.COM'} at 127.0.0.1:55762
I20260809 09:35:20.448201 10161 tablet_service.cc:1514] Processing DeleteTablet for tablet 84b14f1c3fd64b719acbf700e2dbdbe2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 09:35:20 UTC) from {username='kudu', principal='kudu/127.20.125.190@KRBTEST.COM'} at 127.0.0.1:44708
I20260809 09:35:20.448227 10350 tablet_service.cc:1514] Processing DeleteTablet for tablet 84b14f1c3fd64b719acbf700e2dbdbe2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 09:35:20 UTC) from {username='kudu', principal='kudu/127.20.125.190@KRBTEST.COM'} at 127.0.0.1:51514
I20260809 09:35:20.448701 10499 tablet_replica.cc:330] T 84b14f1c3fd64b719acbf700e2dbdbe2 P 26f94d9587f546bebd8f7351a6c30140: stopping tablet replica
I20260809 09:35:20.448872 10498 tablet_replica.cc:330] T 84b14f1c3fd64b719acbf700e2dbdbe2 P c0b76307d73b4cfdba69b55655d1b26d: stopping tablet replica
I20260809 09:35:20.449029 10499 raft_consensus.cc:2229] T 84b14f1c3fd64b719acbf700e2dbdbe2 P 26f94d9587f546bebd8f7351a6c30140 [term 1 LEADER]: Raft consensus shutting down.
I20260809 09:35:20.449235 10498 raft_consensus.cc:2229] T 84b14f1c3fd64b719acbf700e2dbdbe2 P c0b76307d73b4cfdba69b55655d1b26d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 09:35:20.449406 10499 raft_consensus.cc:2258] T 84b14f1c3fd64b719acbf700e2dbdbe2 P 26f94d9587f546bebd8f7351a6c30140 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 09:35:20.449596 20982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu with pid 9822
I20260809 09:35:20.449631 10498 raft_consensus.cc:2258] T 84b14f1c3fd64b719acbf700e2dbdbe2 P c0b76307d73b4cfdba69b55655d1b26d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 09:35:20.454192 10498 ts_tablet_manager.cc:1905] T 84b14f1c3fd64b719acbf700e2dbdbe2 P c0b76307d73b4cfdba69b55655d1b26d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 09:35:20.457042 10499 ts_tablet_manager.cc:1905] T 84b14f1c3fd64b719acbf700e2dbdbe2 P 26f94d9587f546bebd8f7351a6c30140: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 09:35:20.457652 10498 ts_tablet_manager.cc:1918] T 84b14f1c3fd64b719acbf700e2dbdbe2 P c0b76307d73b4cfdba69b55655d1b26d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 09:35:20.458217 10498 log.cc:1196] T 84b14f1c3fd64b719acbf700e2dbdbe2 P c0b76307d73b4cfdba69b55655d1b26d: Deleting WAL directory at /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/minicluster-data/ts-1/wal/wals/84b14f1c3fd64b719acbf700e2dbdbe2
I20260809 09:35:20.458727 10498 ts_tablet_manager.cc:1939] T 84b14f1c3fd64b719acbf700e2dbdbe2 P c0b76307d73b4cfdba69b55655d1b26d: Deleting consensus metadata
I20260809 09:35:20.459884 10499 ts_tablet_manager.cc:1918] T 84b14f1c3fd64b719acbf700e2dbdbe2 P 26f94d9587f546bebd8f7351a6c30140: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 09:35:20.459993 10499 log.cc:1196] T 84b14f1c3fd64b719acbf700e2dbdbe2 P 26f94d9587f546bebd8f7351a6c30140: Deleting WAL directory at /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/minicluster-data/ts-2/wal/wals/84b14f1c3fd64b719acbf700e2dbdbe2
I20260809 09:35:20.460347 9750 catalog_manager.cc:4841] TS c0b76307d73b4cfdba69b55655d1b26d (127.20.125.130:46209): tablet 84b14f1c3fd64b719acbf700e2dbdbe2 (table test-table [id=c7c2ed2765cd424380b0d033ceb24e63]) successfully deleted
I20260809 09:35:20.460773 10499 ts_tablet_manager.cc:1939] T 84b14f1c3fd64b719acbf700e2dbdbe2 P 26f94d9587f546bebd8f7351a6c30140: Deleting consensus metadata
I20260809 09:35:20.461781 9748 catalog_manager.cc:4841] TS 26f94d9587f546bebd8f7351a6c30140 (127.20.125.131:45239): tablet 84b14f1c3fd64b719acbf700e2dbdbe2 (table test-table [id=c7c2ed2765cd424380b0d033ceb24e63]) successfully deleted
W20260809 09:35:20.465915 9748 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.20.125.129:33947 (error 108)
I20260809 09:35:20.466547 20982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu with pid 10053
I20260809 09:35:20.506057 10057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:35:20.505631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.129:33947 (local address 127.0.0.1:55778)
0809 09:35:20.505811 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0809 09:35:20.505936 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.129:33947: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
W20260809 09:35:20.506361 9748 catalog_manager.cc:4584] TS 2128f08f594542018da9bd8141d982c4 (127.20.125.129:33947): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 84b14f1c3fd64b719acbf700e2dbdbe2: Network error: Client connection negotiation failed: client connection to 127.20.125.129:33947: connect: Connection refused (error 111)
I20260809 09:35:20.512951 20982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu with pid 10258
I20260809 09:35:20.527189 20982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu with pid 9723
I20260809 09:35:20.540871 20982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
/tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.125.190:36897
--webserver_interface=127.20.125.190
--webserver_port=34735
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.190.keytab
--principal=kudu/127.20.125.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.125.148:40667
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.125.190:36897
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 09:35:20.654603 10504 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 09:35:20.654873 10504 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 09:35:20.654913 10504 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 09:35:20.658994 10504 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 09:35:20.659046 10504 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 09:35:20.659065 10504 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 09:35:20.659082 10504 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 09:35:20.659097 10504 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 09:35:20.664381 10504 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.125.148:40667
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.125.190:36897
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.190.keytab
--principal=kudu/127.20.125.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.125.190:36897
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.125.190
--webserver_port=34735
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10504
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 08:38:55 UTC on ccycloud.quasar-njjleu.root.comops.site
build id 4575935
I20260809 09:35:20.665525 10504 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 09:35:20.666618 10504 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 09:35:20.675415 10504 server_base.cc:1034] running on GCE node
W20260809 09:35:20.675406 10519 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:35:20.675406 10520 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:35:20.675580 10522 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:35:20.676317 10504 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 09:35:20.677366 10504 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 09:35:20.678546 10504 hybrid_clock.cc:648] HybridClock initialized: now 1786268120678530 us; error 33 us; skew 500 ppm
Aug 09 09:35:20 dist-test-slave-7kzw krb5kdc[8892](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268120, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 09:35:20.681495 10504 init.cc:377] Logged in from keytab as kudu/127.20.125.190@KRBTEST.COM (short username kudu)
I20260809 09:35:20.682606 10504 webserver.cc:460] Webserver started at http://127.20.125.190:34735/ using document root <none> and password file <none>
I20260809 09:35:20.683271 10504 fs_manager.cc:359] Metadata directory not provided
I20260809 09:35:20.683386 10504 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 09:35:20.687707 10504 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260809 09:35:20.690032 10532 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 09:35:20.691443 10504 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 09:35:20.691632 10504 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/minicluster-data/master-0/data,/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/minicluster-data/master-0/wal
uuid: "7a385d41008d4b7aa6e916cf8f8ab796"
format_stamp: "Formatted at 2026-08-09 09:35:18 on dist-test-slave-7kzw"
server_key: "66e740de229301182a61292bfc7e6ea5"
server_key_iv: "714c92229791b2dc8ff32aab8064070f"
server_key_version: "encryptionkey@0"
I20260809 09:35:20.692248 10504 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 09:35:20.716733 10504 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 09:35:20.719733 10504 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 09:35:20.719951 10504 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 09:35:20.727548 10504 rpc_server.cc:307] RPC server started. Bound to: 127.20.125.190:36897
I20260809 09:35:20.727581 10602 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.125.190:36897 every 8 connection(s)
I20260809 09:35:20.728626 10504 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/minicluster-data/master-0/data/info.pb
I20260809 09:35:20.733292 10603 sys_catalog.cc:263] Verifying existing consensus state
I20260809 09:35:20.735167 10603 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7a385d41008d4b7aa6e916cf8f8ab796: Bootstrap starting.
I20260809 09:35:20.738626 20982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu as pid 10504
I20260809 09:35:20.739257 20982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
/tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.125.129:33947
--local_ip_for_outbound_sockets=127.20.125.129
--tserver_master_addrs=127.20.125.190:36897
--webserver_port=37749
--webserver_interface=127.20.125.129
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.129.keytab
--principal=kudu/127.20.125.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.125.148:40667
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 09:35:20.759816 10603 log.cc:824] T 00000000000000000000000000000000 P 7a385d41008d4b7aa6e916cf8f8ab796: Log is configured to *not* fsync() on all Append() calls
I20260809 09:35:20.772794 10603 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7a385d41008d4b7aa6e916cf8f8ab796: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 09:35:20.773270 10603 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7a385d41008d4b7aa6e916cf8f8ab796: Bootstrap complete.
I20260809 09:35:20.778232 10603 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7a385d41008d4b7aa6e916cf8f8ab796 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a385d41008d4b7aa6e916cf8f8ab796" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 36897 } }
I20260809 09:35:20.778752 10603 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7a385d41008d4b7aa6e916cf8f8ab796 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a385d41008d4b7aa6e916cf8f8ab796, State: Initialized, Role: FOLLOWER
I20260809 09:35:20.779295 10603 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7a385d41008d4b7aa6e916cf8f8ab796 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a385d41008d4b7aa6e916cf8f8ab796" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 36897 } }
I20260809 09:35:20.779419 10603 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7a385d41008d4b7aa6e916cf8f8ab796 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 09:35:20.779515 10603 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7a385d41008d4b7aa6e916cf8f8ab796 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 09:35:20.779659 10603 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7a385d41008d4b7aa6e916cf8f8ab796 [term 2 FOLLOWER]: Advancing to term 3
I20260809 09:35:20.781749 10603 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7a385d41008d4b7aa6e916cf8f8ab796 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a385d41008d4b7aa6e916cf8f8ab796" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 36897 } }
I20260809 09:35:20.782120 10603 leader_election.cc:302] T 00000000000000000000000000000000 P 7a385d41008d4b7aa6e916cf8f8ab796 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7a385d41008d4b7aa6e916cf8f8ab796; no voters:
I20260809 09:35:20.782517 10603 leader_election.cc:288] T 00000000000000000000000000000000 P 7a385d41008d4b7aa6e916cf8f8ab796 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260809 09:35:20.782624 10608 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7a385d41008d4b7aa6e916cf8f8ab796 [term 3 FOLLOWER]: Leader election won for term 3
I20260809 09:35:20.782866 10608 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7a385d41008d4b7aa6e916cf8f8ab796 [term 3 LEADER]: Becoming Leader. State: Replica: 7a385d41008d4b7aa6e916cf8f8ab796, State: Running, Role: LEADER
I20260809 09:35:20.783188 10608 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7a385d41008d4b7aa6e916cf8f8ab796 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a385d41008d4b7aa6e916cf8f8ab796" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 36897 } }
I20260809 09:35:20.783887 10603 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7a385d41008d4b7aa6e916cf8f8ab796 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 09:35:20.784192 10608 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7a385d41008d4b7aa6e916cf8f8ab796 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7a385d41008d4b7aa6e916cf8f8ab796. Latest consensus state: current_term: 3 leader_uuid: "7a385d41008d4b7aa6e916cf8f8ab796" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a385d41008d4b7aa6e916cf8f8ab796" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 36897 } } }
I20260809 09:35:20.784327 10608 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7a385d41008d4b7aa6e916cf8f8ab796 [sys.catalog]: This master's current role is: LEADER
I20260809 09:35:20.784754 10609 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7a385d41008d4b7aa6e916cf8f8ab796 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "7a385d41008d4b7aa6e916cf8f8ab796" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a385d41008d4b7aa6e916cf8f8ab796" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 36897 } } }
I20260809 09:35:20.784937 10609 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7a385d41008d4b7aa6e916cf8f8ab796 [sys.catalog]: This master's current role is: LEADER
I20260809 09:35:20.786192 10612 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 09:35:20.791565 10612 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 09:35:20.792205 10612 catalog_manager.cc:1259] Loaded cluster ID: 66f7cc45a5b34d3dbcabda2d042fdf00
I20260809 09:35:20.792276 10612 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260809 09:35:20.793520 10612 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fe4335439ad google::LogMessage::Fail() at ??:0
@ 0x7fe43354688a google::LogMessage::SendToLog() at ??:0
@ 0x7fe4335434de google::LogMessage::Flush() at ??:0
@ 0x7fe433544a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fe43de86fdd _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_ at ??:0
@ 0x7fe43de87d2f kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fe43de84579 _ZZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvENKUlvE_clEv at ??:0
@ 0x7fe43dec8640 _ZNSt17_Function_handlerIFvvEZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7fe4357953d7 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fe435795c83 _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7fe4357973f2 _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7fe435786512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7fe436dca6db start_thread at ??:0
@ 0x7fe43204161f clone at ??:0
W20260809 09:35:20.892189 10606 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 09:35:20.892438 10606 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 09:35:20.892481 10606 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 09:35:20.896476 10606 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 09:35:20.896533 10606 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 09:35:20.896613 10606 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.125.129
I20260809 09:35:20.901825 10606 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.125.148:40667
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.129.keytab
--principal=kudu/127.20.125.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.125.129:33947
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.125.129
--webserver_port=37749
--tserver_master_addrs=127.20.125.190:36897
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10606
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.125.129
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 08:38:55 UTC on ccycloud.quasar-njjleu.root.comops.site
build id 4575935
I20260809 09:35:20.902935 10606 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 09:35:20.904001 10606 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 09:35:20.913817 10657 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:35:20.913805 10659 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:35:20.913805 10661 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:35:20.914321 10606 server_base.cc:1034] running on GCE node
I20260809 09:35:20.914769 10606 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 09:35:20.915443 10606 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 09:35:20.916636 10606 hybrid_clock.cc:648] HybridClock initialized: now 1786268120916614 us; error 42 us; skew 500 ppm
Aug 09 09:35:20 dist-test-slave-7kzw krb5kdc[8892](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268120, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 09:35:20.919667 10606 init.cc:377] Logged in from keytab as kudu/127.20.125.129@KRBTEST.COM (short username kudu)
I20260809 09:35:20.920631 10606 webserver.cc:460] Webserver started at http://127.20.125.129:37749/ using document root <none> and password file <none>
I20260809 09:35:20.921196 10606 fs_manager.cc:359] Metadata directory not provided
I20260809 09:35:20.921260 10606 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 09:35:20.925289 10606 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.001s
I20260809 09:35:20.927520 10673 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 09:35:20.928807 10606 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 09:35:20.928902 10606 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/minicluster-data/ts-0/data,/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/minicluster-data/ts-0/wal
uuid: "2128f08f594542018da9bd8141d982c4"
format_stamp: "Formatted at 2026-08-09 09:35:18 on dist-test-slave-7kzw"
server_key: "96bc8132391916b37f5a364cd436a105"
server_key_iv: "a83b3a7767b9988e4221b52fc64866b3"
server_key_version: "encryptionkey@0"
I20260809 09:35:20.929453 10606 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 09:35:20.950150 10606 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 09:35:20.953706 10606 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 09:35:20.953962 10606 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 09:35:20.954658 10606 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 09:35:20.955852 10684 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260809 09:35:20.961472 10606 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260809 09:35:20.961572 10606 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.000s sys 0.000s
I20260809 09:35:20.961704 10606 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260809 09:35:20.963498 10606 ts_tablet_manager.cc:610] Registered 1 tablets
I20260809 09:35:20.963572 10606 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.001s sys 0.000s
I20260809 09:35:20.963881 10684 tablet_bootstrap.cc:492] T 84b14f1c3fd64b719acbf700e2dbdbe2 P 2128f08f594542018da9bd8141d982c4: Bootstrap starting.
I20260809 09:35:20.974888 10606 rpc_server.cc:307] RPC server started. Bound to: 127.20.125.129:33947
I20260809 09:35:20.974949 10834 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.125.129:33947 every 8 connection(s)
I20260809 09:35:20.975900 10606 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/minicluster-data/ts-0/data/info.pb
I20260809 09:35:20.979081 20982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu as pid 10606
I20260809 09:35:20.979601 20982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
/tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.125.130:46209
--local_ip_for_outbound_sockets=127.20.125.130
--tserver_master_addrs=127.20.125.190:36897
--webserver_port=34923
--webserver_interface=127.20.125.130
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.130.keytab
--principal=kudu/127.20.125.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.125.148:40667
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 09:35:20.989202 10684 log.cc:824] T 84b14f1c3fd64b719acbf700e2dbdbe2 P 2128f08f594542018da9bd8141d982c4: Log is configured to *not* fsync() on all Append() calls
I20260809 09:35:20.996511 10684 tablet_bootstrap.cc:492] T 84b14f1c3fd64b719acbf700e2dbdbe2 P 2128f08f594542018da9bd8141d982c4: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 09:35:20.996932 10684 tablet_bootstrap.cc:492] T 84b14f1c3fd64b719acbf700e2dbdbe2 P 2128f08f594542018da9bd8141d982c4: Bootstrap complete.
I20260809 09:35:20.998137 10684 ts_tablet_manager.cc:1397] T 84b14f1c3fd64b719acbf700e2dbdbe2 P 2128f08f594542018da9bd8141d982c4: Time spent bootstrapping tablet: real 0.035s user 0.020s sys 0.008s
I20260809 09:35:21.000753 10684 raft_consensus.cc:348] T 84b14f1c3fd64b719acbf700e2dbdbe2 P 2128f08f594542018da9bd8141d982c4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26f94d9587f546bebd8f7351a6c30140" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 45239 } } peers { permanent_uuid: "c0b76307d73b4cfdba69b55655d1b26d" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 46209 } } peers { permanent_uuid: "2128f08f594542018da9bd8141d982c4" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 33947 } }
I20260809 09:35:21.001192 10684 raft_consensus.cc:729] T 84b14f1c3fd64b719acbf700e2dbdbe2 P 2128f08f594542018da9bd8141d982c4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2128f08f594542018da9bd8141d982c4, State: Initialized, Role: FOLLOWER
I20260809 09:35:21.001713 10684 consensus_queue.cc:260] T 84b14f1c3fd64b719acbf700e2dbdbe2 P 2128f08f594542018da9bd8141d982c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26f94d9587f546bebd8f7351a6c30140" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 45239 } } peers { permanent_uuid: "c0b76307d73b4cfdba69b55655d1b26d" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 46209 } } peers { permanent_uuid: "2128f08f594542018da9bd8141d982c4" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 33947 } }
I20260809 09:35:21.002786 10684 ts_tablet_manager.cc:1428] T 84b14f1c3fd64b719acbf700e2dbdbe2 P 2128f08f594542018da9bd8141d982c4: Time spent starting tablet: real 0.005s user 0.008s sys 0.000s
W20260809 09:35:21.016100 10837 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 09:35:20.977887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:36897 (local address 127.20.125.129:37957)
0809 09:35:20.978399 (+ 512us) negotiation.cc:107] Waiting for socket to connect
0809 09:35:20.978453 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0809 09:35:20.979658 (+ 1205us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:35:21.015299 (+ 35641us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.190:36897: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":297,"thread_start_us":148,"threads_started":1}
W20260809 09:35:21.016752 10835 heartbeater.cc:644] Failed to heartbeat to 127.20.125.190:36897 (0 consecutive failures): Network error: Failed to ping master at 127.20.125.190:36897: Client connection negotiation failed: client connection to 127.20.125.190:36897: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260809 09:35:21.017513 10837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:35:21.017252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:36897 (local address 127.20.125.129:51099)
0809 09:35:21.017384 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0809 09:35:21.017460 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.190:36897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260809 09:35:21.018239 10837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:35:21.018029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:36897 (local address 127.20.125.129:48463)
0809 09:35:21.018134 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0809 09:35:21.018180 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.190:36897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260809 09:35:21.018509 10835 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 09:35:21.109237 10838 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 09:35:21.109562 10838 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 09:35:21.109645 10838 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 09:35:21.113778 10838 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 09:35:21.113869 10838 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 09:35:21.113962 10838 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.125.130
I20260809 09:35:21.119138 10838 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.125.148:40667
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.130.keytab
--principal=kudu/127.20.125.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.125.130:46209
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.125.130
--webserver_port=34923
--tserver_master_addrs=127.20.125.190:36897
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10838
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.125.130
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 08:38:55 UTC on ccycloud.quasar-njjleu.root.comops.site
build id 4575935
I20260809 09:35:21.120190 10838 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 09:35:21.121232 10838 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 09:35:21.131073 10858 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:35:21.131049 10856 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:35:21.131103 10855 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:35:21.131335 10838 server_base.cc:1034] running on GCE node
I20260809 09:35:21.131907 10838 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 09:35:21.132611 10838 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 09:35:21.133797 10838 hybrid_clock.cc:648] HybridClock initialized: now 1786268121133765 us; error 44 us; skew 500 ppm
Aug 09 09:35:21 dist-test-slave-7kzw krb5kdc[8892](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268121, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 09:35:21.136878 10838 init.cc:377] Logged in from keytab as kudu/127.20.125.130@KRBTEST.COM (short username kudu)
I20260809 09:35:21.137741 10838 webserver.cc:460] Webserver started at http://127.20.125.130:34923/ using document root <none> and password file <none>
I20260809 09:35:21.138275 10838 fs_manager.cc:359] Metadata directory not provided
I20260809 09:35:21.138336 10838 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 09:35:21.142282 10838 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260809 09:35:21.144418 10868 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 09:35:21.145648 10838 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 09:35:21.145908 10838 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/minicluster-data/ts-1/data,/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/minicluster-data/ts-1/wal
uuid: "c0b76307d73b4cfdba69b55655d1b26d"
format_stamp: "Formatted at 2026-08-09 09:35:18 on dist-test-slave-7kzw"
server_key: "50584686f8518c032d7ff83ee5686320"
server_key_iv: "ce09d747ffff61d08f59d38ae4903162"
server_key_version: "encryptionkey@0"
I20260809 09:35:21.146437 10838 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 09:35:21.157573 10838 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 09:35:21.161037 10838 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 09:35:21.161250 10838 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 09:35:21.161891 10838 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 09:35:21.162801 10838 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 09:35:21.162850 10838 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 09:35:21.162892 10838 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 09:35:21.162951 10838 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 09:35:21.172119 10838 rpc_server.cc:307] RPC server started. Bound to: 127.20.125.130:46209
I20260809 09:35:21.172149 11034 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.125.130:46209 every 8 connection(s)
I20260809 09:35:21.173193 10838 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/minicluster-data/ts-1/data/info.pb
I20260809 09:35:21.177146 11038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:35:21.175465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:36897 (local address 127.20.125.130:48545)
0809 09:35:21.175956 (+ 491us) negotiation.cc:107] Waiting for socket to connect
0809 09:35:21.176390 (+ 434us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.190:36897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":291,"thread_start_us":113,"threads_started":1}
W20260809 09:35:21.177636 11036 heartbeater.cc:644] Failed to heartbeat to 127.20.125.190:36897 (0 consecutive failures): Network error: Failed to ping master at 127.20.125.190:36897: Client connection negotiation failed: client connection to 127.20.125.190:36897: connect: Connection refused (error 111)
I20260809 09:35:21.178232 11038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:35:21.178019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:36897 (local address 127.20.125.130:56699)
0809 09:35:21.178127 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0809 09:35:21.178176 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.190:36897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260809 09:35:21.178907 11038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:35:21.178702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:36897 (local address 127.20.125.130:46741)
0809 09:35:21.178820 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0809 09:35:21.178862 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.190:36897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260809 09:35:21.179159 11036 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260809 09:35:21.182722 20982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu as pid 10838
I20260809 09:35:21.183128 20982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
/tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.125.131:45239
--local_ip_for_outbound_sockets=127.20.125.131
--tserver_master_addrs=127.20.125.190:36897
--webserver_port=41433
--webserver_interface=127.20.125.131
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.131.keytab
--principal=kudu/127.20.125.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.125.148:40667
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 09:35:21.296880 11039 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 09:35:21.297127 11039 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 09:35:21.297168 11039 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 09:35:21.301115 11039 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 09:35:21.301175 11039 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 09:35:21.301231 11039 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.125.131
I20260809 09:35:21.306360 11039 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.125.148:40667
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.131.keytab
--principal=kudu/127.20.125.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.125.131:45239
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.125.131
--webserver_port=41433
--tserver_master_addrs=127.20.125.190:36897
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11039
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.125.131
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 08:38:55 UTC on ccycloud.quasar-njjleu.root.comops.site
build id 4575935
I20260809 09:35:21.307461 11039 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 09:35:21.308496 11039 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 09:35:21.318483 11051 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:35:21.318492 11049 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:35:21.318651 11039 server_base.cc:1034] running on GCE node
W20260809 09:35:21.318601 11048 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:35:21.319244 11039 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 09:35:21.319988 11039 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 09:35:21.321175 11039 hybrid_clock.cc:648] HybridClock initialized: now 1786268121321146 us; error 41 us; skew 500 ppm
Aug 09 09:35:21 dist-test-slave-7kzw krb5kdc[8892](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268121, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 09:35:21.324138 11039 init.cc:377] Logged in from keytab as kudu/127.20.125.131@KRBTEST.COM (short username kudu)
I20260809 09:35:21.325037 11039 webserver.cc:460] Webserver started at http://127.20.125.131:41433/ using document root <none> and password file <none>
I20260809 09:35:21.325596 11039 fs_manager.cc:359] Metadata directory not provided
I20260809 09:35:21.325719 11039 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 09:35:21.329797 11039 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260809 09:35:21.332069 11065 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 09:35:21.333333 11039 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 09:35:21.333467 11039 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/minicluster-data/ts-2/data,/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/minicluster-data/ts-2/wal
uuid: "26f94d9587f546bebd8f7351a6c30140"
format_stamp: "Formatted at 2026-08-09 09:35:19 on dist-test-slave-7kzw"
server_key: "1d155273c6f05f49880d6d2b74b34b6c"
server_key_iv: "e9ff8f13dc172def69b26f94e67bf0bc"
server_key_version: "encryptionkey@0"
I20260809 09:35:21.334116 11039 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 09:35:21.346690 11039 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 09:35:21.349907 11039 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 09:35:21.350119 11039 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 09:35:21.350889 11039 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 09:35:21.351737 11039 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 09:35:21.351815 11039 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 09:35:21.351883 11039 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 09:35:21.351938 11039 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 09:35:21.361379 11039 rpc_server.cc:307] RPC server started. Bound to: 127.20.125.131:45239
I20260809 09:35:21.361435 11246 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.125.131:45239 every 8 connection(s)
I20260809 09:35:21.362380 11039 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786267941944493-20982-0/minicluster-data/ts-2/data/info.pb
I20260809 09:35:21.366053 11249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:35:21.364555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:36897 (local address 127.20.125.131:53479)
0809 09:35:21.365243 (+ 688us) negotiation.cc:107] Waiting for socket to connect
0809 09:35:21.365361 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.190:36897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":488,"thread_start_us":110,"threads_started":1}
W20260809 09:35:21.366647 11247 heartbeater.cc:644] Failed to heartbeat to 127.20.125.190:36897 (0 consecutive failures): Network error: Failed to ping master at 127.20.125.190:36897: Client connection negotiation failed: client connection to 127.20.125.190:36897: connect: Connection refused (error 111)
I20260809 09:35:21.367311 11249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:35:21.367030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:36897 (local address 127.20.125.131:35519)
0809 09:35:21.367177 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0809 09:35:21.367234 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.190:36897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260809 09:35:21.368093 11249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:35:21.367856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:36897 (local address 127.20.125.131:48011)
0809 09:35:21.367981 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0809 09:35:21.368040 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.190:36897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260809 09:35:21.368307 11247 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260809 09:35:21.370872 20982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu as pid 11039
I20260809 09:35:22.019634 11251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:35:22.019165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:36897 (local address 127.20.125.129:37127)
0809 09:35:22.019448 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0809 09:35:22.019512 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.190:36897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":76,"threads_started":1}
I20260809 09:35:22.180200 11252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:35:22.179773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:36897 (local address 127.20.125.130:44575)
0809 09:35:22.180031 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0809 09:35:22.180091 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.190:36897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":73,"threads_started":1}
I20260809 09:35:22.369328 11253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:35:22.368899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:36897 (local address 127.20.125.131:48567)
0809 09:35:22.369164 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0809 09:35:22.369223 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.190:36897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":71,"threads_started":1}
I20260809 09:35:22.580243 11254 raft_consensus.cc:482] T 84b14f1c3fd64b719acbf700e2dbdbe2 P 2128f08f594542018da9bd8141d982c4 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 09:35:22.580449 11254 raft_consensus.cc:504] T 84b14f1c3fd64b719acbf700e2dbdbe2 P 2128f08f594542018da9bd8141d982c4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26f94d9587f546bebd8f7351a6c30140" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 45239 } } peers { permanent_uuid: "c0b76307d73b4cfdba69b55655d1b26d" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 46209 } } peers { permanent_uuid: "2128f08f594542018da9bd8141d982c4" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 33947 } }
I20260809 09:35:22.581550 11254 leader_election.cc:288] T 84b14f1c3fd64b719acbf700e2dbdbe2 P 2128f08f594542018da9bd8141d982c4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 26f94d9587f546bebd8f7351a6c30140 (127.20.125.131:45239), c0b76307d73b4cfdba69b55655d1b26d (127.20.125.130:46209)
Aug 09 09:35:22 dist-test-slave-7kzw krb5kdc[8892](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268120, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.129@KRBTEST.COM for kudu/127.20.125.130@KRBTEST.COM
Aug 09 09:35:22 dist-test-slave-7kzw krb5kdc[8892](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268120, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.129@KRBTEST.COM for kudu/127.20.125.131@KRBTEST.COM
I20260809 09:35:22.600394 11256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:35:22.581816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.131:45239 (local address 127.20.125.129:56361)
0809 09:35:22.582188 (+ 372us) negotiation.cc:107] Waiting for socket to connect
0809 09:35:22.582205 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 09:35:22.582324 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:35:22.584205 (+ 1881us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:35:22.584212 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:35:22.584770 (+ 558us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:35:22.585261 (+ 491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:35:22.585276 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:35:22.586729 (+ 1453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:35:22.586735 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:35:22.587520 (+ 785us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:35:22.587529 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:35:22.588316 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:35:22.588320 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:35:22.588439 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:35:22.592826 (+ 4387us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 09:35:22.592852 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:35:22.595255 (+ 2403us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 09:35:22.597570 (+ 2315us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:35:22.597578 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:35:22.597596 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:35:22.598013 (+ 417us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:35:22.598429 (+ 416us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:35:22.598432 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:35:22.598434 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:35:22.598680 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:35:22.599341 (+ 661us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 09:35:22.599348 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 09:35:22.599637 (+ 289us) client_negotiation.cc:769] Sending connection context
0809 09:35:22.599827 (+ 190us) client_negotiation.cc:240] Negotiation successful
0809 09:35:22.600176 (+ 349us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":293,"mutex_wait_us":78,"spinlock_wait_cycles":56576,"thread_start_us":84,"threads_started":1}
I20260809 09:35:22.600407 11257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:35:22.582012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.129:56887 (local address 127.20.125.130:46209)
0809 09:35:22.582501 (+ 489us) server_negotiation.cc:206] Beginning negotiation
0809 09:35:22.582531 (+ 30us) server_negotiation.cc:399] Waiting for connection header
0809 09:35:22.582582 (+ 51us) server_negotiation.cc:407] Connection header received
0809 09:35:22.583374 (+ 792us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:35:22.583383 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:35:22.583847 (+ 464us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:35:22.584161 (+ 314us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:35:22.585445 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:35:22.586422 (+ 977us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:35:22.587657 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:35:22.588126 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:35:22.588309 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:35:22.595507 (+ 7198us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:35:22.595513 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:35:22.595525 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:35:22.595555 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:35:22.597640 (+ 2085us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:35:22.598181 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:35:22.598186 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:35:22.598190 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:35:22.598321 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:35:22.598784 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:35:22.598788 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:35:22.598789 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:35:22.599215 (+ 426us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:35:22.599365 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0809 09:35:22.599957 (+ 592us) server_negotiation.cc:299] Negotiation successful
0809 09:35:22.600232 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":317,"thread_start_us":81,"threads_started":1}
I20260809 09:35:22.600435 11261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:35:22.581823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.130:46209 (local address 127.20.125.129:56887)
0809 09:35:22.582359 (+ 536us) negotiation.cc:107] Waiting for socket to connect
0809 09:35:22.582375 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 09:35:22.582502 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:35:22.584207 (+ 1705us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:35:22.584213 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:35:22.584769 (+ 556us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:35:22.585261 (+ 492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:35:22.585275 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:35:22.586614 (+ 1339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:35:22.586617 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:35:22.587519 (+ 902us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:35:22.587528 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:35:22.588258 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:35:22.588261 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:35:22.588435 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:35:22.592833 (+ 4398us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 09:35:22.592853 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:35:22.595238 (+ 2385us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 09:35:22.597772 (+ 2534us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:35:22.597775 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:35:22.597777 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:35:22.598034 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:35:22.598437 (+ 403us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:35:22.598440 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:35:22.598442 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:35:22.598677 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:35:22.599341 (+ 664us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 09:35:22.599348 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 09:35:22.599634 (+ 286us) client_negotiation.cc:769] Sending connection context
0809 09:35:22.599827 (+ 193us) client_negotiation.cc:240] Negotiation successful
0809 09:35:22.600264 (+ 437us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":471,"mutex_wait_us":409,"spinlock_wait_cycles":10880,"thread_start_us":160,"threads_started":1}
I20260809 09:35:22.600445 11259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:35:22.581991 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.129:56361 (local address 127.20.125.131:45239)
0809 09:35:22.582518 (+ 527us) server_negotiation.cc:206] Beginning negotiation
0809 09:35:22.582551 (+ 33us) server_negotiation.cc:399] Waiting for connection header
0809 09:35:22.582587 (+ 36us) server_negotiation.cc:407] Connection header received
0809 09:35:22.583457 (+ 870us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:35:22.583468 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:35:22.583881 (+ 413us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:35:22.584203 (+ 322us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:35:22.585450 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:35:22.586592 (+ 1142us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:35:22.587670 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:35:22.588165 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:35:22.588351 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:35:22.595506 (+ 7155us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:35:22.595511 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:35:22.595524 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:35:22.595556 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:35:22.597433 (+ 1877us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:35:22.598171 (+ 738us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:35:22.598178 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:35:22.598185 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:35:22.598322 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:35:22.598786 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:35:22.598789 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:35:22.598791 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:35:22.599215 (+ 424us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:35:22.599365 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0809 09:35:22.599951 (+ 586us) server_negotiation.cc:299] Negotiation successful
0809 09:35:22.600252 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":355,"thread_start_us":122,"threads_started":1}
W20260809 09:35:22.603503 10676 leader_election.cc:341] T 84b14f1c3fd64b719acbf700e2dbdbe2 P 2128f08f594542018da9bd8141d982c4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 26f94d9587f546bebd8f7351a6c30140 (127.20.125.131:45239): Not found: Tablet not found: 84b14f1c3fd64b719acbf700e2dbdbe2
W20260809 09:35:22.603684 10677 leader_election.cc:341] T 84b14f1c3fd64b719acbf700e2dbdbe2 P 2128f08f594542018da9bd8141d982c4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer c0b76307d73b4cfdba69b55655d1b26d (127.20.125.130:46209): Not found: Tablet not found: 84b14f1c3fd64b719acbf700e2dbdbe2
I20260809 09:35:22.603760 10676 leader_election.cc:302] T 84b14f1c3fd64b719acbf700e2dbdbe2 P 2128f08f594542018da9bd8141d982c4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2128f08f594542018da9bd8141d982c4; no voters: 26f94d9587f546bebd8f7351a6c30140, c0b76307d73b4cfdba69b55655d1b26d
I20260809 09:35:22.604071 11254 raft_consensus.cc:2726] T 84b14f1c3fd64b719acbf700e2dbdbe2 P 2128f08f594542018da9bd8141d982c4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 09:35:23.020850 11261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:35:23.020473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:36897 (local address 127.20.125.129:55293)
0809 09:35:23.020674 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0809 09:35:23.020749 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.190:36897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102}
I20260809 09:35:23.181452 11262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:35:23.181008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:36897 (local address 127.20.125.130:44929)
0809 09:35:23.181298 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0809 09:35:23.181356 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.190:36897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":82,"threads_started":1}
I20260809 09:35:23.370633 11263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:35:23.370142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:36897 (local address 127.20.125.131:57787)
0809 09:35:23.370391 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0809 09:35:23.370450 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.190:36897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":76,"threads_started":1}
I20260809 09:35:24.022526 11265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:35:24.021926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:36897 (local address 127.20.125.129:34341)
0809 09:35:24.022296 (+ 370us) negotiation.cc:107] Waiting for socket to connect
0809 09:35:24.022356 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.190:36897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":285,"thread_start_us":110,"threads_started":1}
I20260809 09:35:24.182796 11267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:35:24.182279 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:36897 (local address 127.20.125.130:54123)
0809 09:35:24.182625 (+ 346us) negotiation.cc:107] Waiting for socket to connect
0809 09:35:24.182689 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.190:36897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":260,"thread_start_us":81,"threads_started":1}
I20260809 09:35:24.371927 11268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:35:24.371470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:36897 (local address 127.20.125.131:36387)
0809 09:35:24.371768 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0809 09:35:24.371832 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.190:36897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":78,"threads_started":1}
I20260809 09:35:24.544028 11270 raft_consensus.cc:482] T 84b14f1c3fd64b719acbf700e2dbdbe2 P 2128f08f594542018da9bd8141d982c4 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 09:35:24.544170 11270 raft_consensus.cc:504] T 84b14f1c3fd64b719acbf700e2dbdbe2 P 2128f08f594542018da9bd8141d982c4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26f94d9587f546bebd8f7351a6c30140" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 45239 } } peers { permanent_uuid: "c0b76307d73b4cfdba69b55655d1b26d" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 46209 } } peers { permanent_uuid: "2128f08f594542018da9bd8141d982c4" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 33947 } }
I20260809 09:35:24.544589 11270 leader_election.cc:288] T 84b14f1c3fd64b719acbf700e2dbdbe2 P 2128f08f594542018da9bd8141d982c4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 26f94d9587f546bebd8f7351a6c30140 (127.20.125.131:45239), c0b76307d73b4cfdba69b55655d1b26d (127.20.125.130:46209)
W20260809 09:35:24.545612 10677 leader_election.cc:341] T 84b14f1c3fd64b719acbf700e2dbdbe2 P 2128f08f594542018da9bd8141d982c4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer c0b76307d73b4cfdba69b55655d1b26d (127.20.125.130:46209): Not found: Tablet not found: 84b14f1c3fd64b719acbf700e2dbdbe2
W20260809 09:35:24.545804 10676 leader_election.cc:341] T 84b14f1c3fd64b719acbf700e2dbdbe2 P 2128f08f594542018da9bd8141d982c4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 26f94d9587f546bebd8f7351a6c30140 (127.20.125.131:45239): Not found: Tablet not found: 84b14f1c3fd64b719acbf700e2dbdbe2
I20260809 09:35:24.545931 10676 leader_election.cc:302] T 84b14f1c3fd64b719acbf700e2dbdbe2 P 2128f08f594542018da9bd8141d982c4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2128f08f594542018da9bd8141d982c4; no voters: 26f94d9587f546bebd8f7351a6c30140, c0b76307d73b4cfdba69b55655d1b26d
I20260809 09:35:24.546103 11270 raft_consensus.cc:2726] T 84b14f1c3fd64b719acbf700e2dbdbe2 P 2128f08f594542018da9bd8141d982c4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 09:35:25.023904 11271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:35:25.023399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:36897 (local address 127.20.125.129:48897)
0809 09:35:25.023710 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0809 09:35:25.023772 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.190:36897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":86,"threads_started":1}
I20260809 09:35:25.184175 11273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:35:25.183691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:36897 (local address 127.20.125.130:57227)
0809 09:35:25.183990 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0809 09:35:25.184070 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.190:36897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":83,"threads_started":1}
I20260809 09:35:25.373265 11274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:35:25.372823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:36897 (local address 127.20.125.131:36885)
0809 09:35:25.373090 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0809 09:35:25.373152 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.190:36897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":78,"threads_started":1}
I20260809 09:35:26.025599 11275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:35:26.025000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:36897 (local address 127.20.125.129:45557)
0809 09:35:26.025382 (+ 382us) negotiation.cc:107] Waiting for socket to connect
0809 09:35:26.025453 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.190:36897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":290,"thread_start_us":86,"threads_started":1}
I20260809 09:35:26.185567 11276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:35:26.185076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:36897 (local address 127.20.125.130:35355)
0809 09:35:26.185375 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0809 09:35:26.185436 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.190:36897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":86,"threads_started":1}
I20260809 09:35:26.374666 11277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:35:26.374209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:36897 (local address 127.20.125.131:53053)
0809 09:35:26.374497 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0809 09:35:26.374568 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.190:36897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":96,"threads_started":1}
I20260809 09:35:27.027204 11278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:35:27.026677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:36897 (local address 127.20.125.129:56969)
0809 09:35:27.027020 (+ 343us) negotiation.cc:107] Waiting for socket to connect
0809 09:35:27.027087 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.190:36897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":87,"threads_started":1}
I20260809 09:35:27.186911 11279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:35:27.186407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:36897 (local address 127.20.125.130:37639)
0809 09:35:27.186754 (+ 347us) negotiation.cc:107] Waiting for socket to connect
0809 09:35:27.186811 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.190:36897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":273,"thread_start_us":156,"threads_started":1}
I20260809 09:35:27.375877 11280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:35:27.375460 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:36897 (local address 127.20.125.131:39201)
0809 09:35:27.375720 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0809 09:35:27.375777 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.190:36897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":82,"threads_started":1}
I20260809 09:35:27.760649 11281 raft_consensus.cc:482] T 84b14f1c3fd64b719acbf700e2dbdbe2 P 2128f08f594542018da9bd8141d982c4 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 09:35:27.760826 11281 raft_consensus.cc:504] T 84b14f1c3fd64b719acbf700e2dbdbe2 P 2128f08f594542018da9bd8141d982c4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26f94d9587f546bebd8f7351a6c30140" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 45239 } } peers { permanent_uuid: "c0b76307d73b4cfdba69b55655d1b26d" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 46209 } } peers { permanent_uuid: "2128f08f594542018da9bd8141d982c4" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 33947 } }
I20260809 09:35:27.761234 11281 leader_election.cc:288] T 84b14f1c3fd64b719acbf700e2dbdbe2 P 2128f08f594542018da9bd8141d982c4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 26f94d9587f546bebd8f7351a6c30140 (127.20.125.131:45239), c0b76307d73b4cfdba69b55655d1b26d (127.20.125.130:46209)
W20260809 09:35:27.762279 10677 leader_election.cc:341] T 84b14f1c3fd64b719acbf700e2dbdbe2 P 2128f08f594542018da9bd8141d982c4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer c0b76307d73b4cfdba69b55655d1b26d (127.20.125.130:46209): Not found: Tablet not found: 84b14f1c3fd64b719acbf700e2dbdbe2
W20260809 09:35:27.762419 10676 leader_election.cc:341] T 84b14f1c3fd64b719acbf700e2dbdbe2 P 2128f08f594542018da9bd8141d982c4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 26f94d9587f546bebd8f7351a6c30140 (127.20.125.131:45239): Not found: Tablet not found: 84b14f1c3fd64b719acbf700e2dbdbe2
I20260809 09:35:27.762578 10676 leader_election.cc:302] T 84b14f1c3fd64b719acbf700e2dbdbe2 P 2128f08f594542018da9bd8141d982c4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2128f08f594542018da9bd8141d982c4; no voters: 26f94d9587f546bebd8f7351a6c30140, c0b76307d73b4cfdba69b55655d1b26d
I20260809 09:35:27.762787 11281 raft_consensus.cc:2726] T 84b14f1c3fd64b719acbf700e2dbdbe2 P 2128f08f594542018da9bd8141d982c4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 09:35:28.028543 11282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:35:28.028081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:36897 (local address 127.20.125.129:49077)
0809 09:35:28.028372 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0809 09:35:28.028435 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.190:36897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":107,"threads_started":1}
I20260809 09:35:28.188571 11283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:35:28.187944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:36897 (local address 127.20.125.130:33671)
0809 09:35:28.188356 (+ 412us) negotiation.cc:107] Waiting for socket to connect
0809 09:35:28.188433 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.190:36897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":316,"thread_start_us":80,"threads_started":1}
I20260809 09:35:28.377147 11284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:35:28.376640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:36897 (local address 127.20.125.131:50879)
0809 09:35:28.376962 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0809 09:35:28.377035 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.190:36897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":246,"thread_start_us":74,"threads_started":1}
I20260809 09:35:29.029883 11285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:35:29.029411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:36897 (local address 127.20.125.129:33969)
0809 09:35:29.029715 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0809 09:35:29.029778 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.190:36897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":221,"thread_start_us":86,"threads_started":1}
I20260809 09:35:29.189931 11288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:35:29.189433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:36897 (local address 127.20.125.130:39789)
0809 09:35:29.189769 (+ 336us) negotiation.cc:107] Waiting for socket to connect
0809 09:35:29.189827 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.190:36897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":260,"thread_start_us":85,"threads_started":1}
I20260809 09:35:29.378566 11289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:35:29.378042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:36897 (local address 127.20.125.131:54563)
0809 09:35:29.378327 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0809 09:35:29.378385 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.190:36897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":101,"threads_started":1}
I20260809 09:35:30.031361 11290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:35:30.030874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:36897 (local address 127.20.125.129:34611)
0809 09:35:30.031189 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0809 09:35:30.031255 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.190:36897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":231,"thread_start_us":88,"threads_started":1}
I20260809 09:35:30.191478 11291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:35:30.190964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:36897 (local address 127.20.125.130:35185)
0809 09:35:30.191301 (+ 337us) negotiation.cc:107] Waiting for socket to connect
0809 09:35:30.191369 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.190:36897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":91,"threads_started":1}
I20260809 09:35:30.381494 11292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:35:30.380940 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:36897 (local address 127.20.125.131:47407)
0809 09:35:30.381265 (+ 325us) negotiation.cc:107] Waiting for socket to connect
0809 09:35:30.381352 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.190:36897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":73,"threads_started":1}
I20260809 09:35:31.032881 11293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:35:31.032313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:36897 (local address 127.20.125.129:52121)
0809 09:35:31.032617 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0809 09:35:31.032688 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.190:36897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":79,"threads_started":1}
I20260809 09:35:31.192952 11297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:35:31.192451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:36897 (local address 127.20.125.130:48587)
0809 09:35:31.192786 (+ 335us) negotiation.cc:107] Waiting for socket to connect
0809 09:35:31.192850 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.190:36897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":85,"threads_started":1}
I20260809 09:35:31.382985 11298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:35:31.382503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:36897 (local address 127.20.125.131:56677)
0809 09:35:31.382826 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0809 09:35:31.382881 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.190:36897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":101,"threads_started":1}
I20260809 09:35:32.034147 11300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:35:32.033725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:36897 (local address 127.20.125.129:47555)
0809 09:35:32.033980 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0809 09:35:32.034041 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.190:36897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":74,"threads_started":1}
I20260809 09:35:32.046838 11301 raft_consensus.cc:482] T 84b14f1c3fd64b719acbf700e2dbdbe2 P 2128f08f594542018da9bd8141d982c4 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 09:35:32.046969 11301 raft_consensus.cc:504] T 84b14f1c3fd64b719acbf700e2dbdbe2 P 2128f08f594542018da9bd8141d982c4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26f94d9587f546bebd8f7351a6c30140" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 45239 } } peers { permanent_uuid: "c0b76307d73b4cfdba69b55655d1b26d" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 46209 } } peers { permanent_uuid: "2128f08f594542018da9bd8141d982c4" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 33947 } }
I20260809 09:35:32.047358 11301 leader_election.cc:288] T 84b14f1c3fd64b719acbf700e2dbdbe2 P 2128f08f594542018da9bd8141d982c4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 26f94d9587f546bebd8f7351a6c30140 (127.20.125.131:45239), c0b76307d73b4cfdba69b55655d1b26d (127.20.125.130:46209)
W20260809 09:35:32.048216 10676 leader_election.cc:341] T 84b14f1c3fd64b719acbf700e2dbdbe2 P 2128f08f594542018da9bd8141d982c4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 26f94d9587f546bebd8f7351a6c30140 (127.20.125.131:45239): Not found: Tablet not found: 84b14f1c3fd64b719acbf700e2dbdbe2
W20260809 09:35:32.048383 10677 leader_election.cc:341] T 84b14f1c3fd64b719acbf700e2dbdbe2 P 2128f08f594542018da9bd8141d982c4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer c0b76307d73b4cfdba69b55655d1b26d (127.20.125.130:46209): Not found: Tablet not found: 84b14f1c3fd64b719acbf700e2dbdbe2
I20260809 09:35:32.048463 10677 leader_election.cc:302] T 84b14f1c3fd64b719acbf700e2dbdbe2 P 2128f08f594542018da9bd8141d982c4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2128f08f594542018da9bd8141d982c4; no voters: 26f94d9587f546bebd8f7351a6c30140, c0b76307d73b4cfdba69b55655d1b26d
I20260809 09:35:32.048625 11301 raft_consensus.cc:2726] T 84b14f1c3fd64b719acbf700e2dbdbe2 P 2128f08f594542018da9bd8141d982c4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 09:35:32.194375 11303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:35:32.193889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:36897 (local address 127.20.125.130:42477)
0809 09:35:32.194192 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0809 09:35:32.194263 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.190:36897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":88,"threads_started":1}
I20260809 09:35:32.384225 11304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:35:32.383777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:36897 (local address 127.20.125.131:36363)
0809 09:35:32.384062 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0809 09:35:32.384121 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.190:36897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":79,"threads_started":1}
I20260809 09:35:33.035579 11305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:35:33.035052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:36897 (local address 127.20.125.129:35599)
0809 09:35:33.035374 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0809 09:35:33.035452 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.190:36897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":231,"thread_start_us":101,"threads_started":1}
I20260809 09:35:33.195649 11306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:35:33.195246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:36897 (local address 127.20.125.130:57753)
0809 09:35:33.195492 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0809 09:35:33.195549 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.190:36897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":79,"threads_started":1}
I20260809 09:35:33.385509 11308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:35:33.385060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:36897 (local address 127.20.125.131:43949)
0809 09:35:33.385337 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0809 09:35:33.385400 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.190:36897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":77,"threads_started":1}
I20260809 09:35:34.037230 11309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:35:34.036706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:36897 (local address 127.20.125.129:52179)
0809 09:35:34.037016 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0809 09:35:34.037091 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.190:36897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":84,"threads_started":1}
I20260809 09:35:34.197206 11312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:35:34.196703 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:36897 (local address 127.20.125.130:38221)
0809 09:35:34.197028 (+ 325us) negotiation.cc:107] Waiting for socket to connect
0809 09:35:34.197100 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.190:36897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":236,"thread_start_us":93,"threads_started":1}
I20260809 09:35:34.387068 11314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:35:34.386549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:36897 (local address 127.20.125.131:45991)
0809 09:35:34.386873 (+ 324us) negotiation.cc:107] Waiting for socket to connect
0809 09:35:34.386946 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.190:36897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":233,"thread_start_us":80,"threads_started":1}
I20260809 09:35:35.038656 11315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:35:35.038083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:36897 (local address 127.20.125.129:34243)
0809 09:35:35.038426 (+ 343us) negotiation.cc:107] Waiting for socket to connect
0809 09:35:35.038537 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.190:36897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":82,"threads_started":1}
I20260809 09:35:35.198650 11317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:35:35.198105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:36897 (local address 127.20.125.130:45195)
0809 09:35:35.198416 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0809 09:35:35.198529 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.190:36897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":131,"threads_started":1}
I20260809 09:35:35.388393 11319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:35:35.387930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:36897 (local address 127.20.125.131:46227)
0809 09:35:35.388229 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0809 09:35:35.388288 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.190:36897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":84,"threads_started":1}
I20260809 09:35:35.512989 11320 raft_consensus.cc:482] T 84b14f1c3fd64b719acbf700e2dbdbe2 P 2128f08f594542018da9bd8141d982c4 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 09:35:35.513144 11320 raft_consensus.cc:504] T 84b14f1c3fd64b719acbf700e2dbdbe2 P 2128f08f594542018da9bd8141d982c4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26f94d9587f546bebd8f7351a6c30140" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 45239 } } peers { permanent_uuid: "c0b76307d73b4cfdba69b55655d1b26d" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 46209 } } peers { permanent_uuid: "2128f08f594542018da9bd8141d982c4" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 33947 } }
I20260809 09:35:35.513669 11320 leader_election.cc:288] T 84b14f1c3fd64b719acbf700e2dbdbe2 P 2128f08f594542018da9bd8141d982c4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 26f94d9587f546bebd8f7351a6c30140 (127.20.125.131:45239), c0b76307d73b4cfdba69b55655d1b26d (127.20.125.130:46209)
W20260809 09:35:35.514783 10676 leader_election.cc:341] T 84b14f1c3fd64b719acbf700e2dbdbe2 P 2128f08f594542018da9bd8141d982c4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 26f94d9587f546bebd8f7351a6c30140 (127.20.125.131:45239): Not found: Tablet not found: 84b14f1c3fd64b719acbf700e2dbdbe2
W20260809 09:35:35.514955 10677 leader_election.cc:341] T 84b14f1c3fd64b719acbf700e2dbdbe2 P 2128f08f594542018da9bd8141d982c4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer c0b76307d73b4cfdba69b55655d1b26d (127.20.125.130:46209): Not found: Tablet not found: 84b14f1c3fd64b719acbf700e2dbdbe2
I20260809 09:35:35.515043 10677 leader_election.cc:302] T 84b14f1c3fd64b719acbf700e2dbdbe2 P 2128f08f594542018da9bd8141d982c4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2128f08f594542018da9bd8141d982c4; no voters: 26f94d9587f546bebd8f7351a6c30140, c0b76307d73b4cfdba69b55655d1b26d
I20260809 09:35:35.515206 11320 raft_consensus.cc:2726] T 84b14f1c3fd64b719acbf700e2dbdbe2 P 2128f08f594542018da9bd8141d982c4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 09:35:36.040131 11322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:35:36.039653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:36897 (local address 127.20.125.129:46043)
0809 09:35:36.039911 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0809 09:35:36.039968 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.190:36897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":83,"threads_started":1}
I20260809 09:35:36.200078 11324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:35:36.199569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:36897 (local address 127.20.125.130:59033)
0809 09:35:36.199903 (+ 334us) negotiation.cc:107] Waiting for socket to connect
0809 09:35:36.199976 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.190:36897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":251,"thread_start_us":88,"threads_started":1}
I20260809 09:35:36.376507 20982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu with pid 10606
I20260809 09:35:36.396145 11328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:35:36.395248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:36897 (local address 127.20.125.131:57151)
0809 09:35:36.395899 (+ 651us) negotiation.cc:107] Waiting for socket to connect
0809 09:35:36.395981 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.190:36897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":549,"thread_start_us":327,"threads_started":1}
I20260809 09:35:36.403748 20982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu with pid 10838
I20260809 09:35:36.415066 20982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu with pid 11039
2026-08-09T09:35:36Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (19630 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 09:35:36 dist-test-slave-7kzw krb5kdc[11335](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 09:35:36 dist-test-slave-7kzw krb5kdc[11335](info): set up 2 sockets
Aug 09 09:35:36 dist-test-slave-7kzw krb5kdc[11335](info): commencing operation
krb5kdc: starting...
W20260809 09:35:37.727447 20982 mini_kdc.cc:121] Time spent starting KDC: real 1.275s user 0.003s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/krb5kdc/test-user.keytab.
Aug 09 09:35:37 dist-test-slave-7kzw krb5kdc[11335](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268137, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T09:35:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T09:35:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.125.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.125.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.125.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.190.keytab.
Entry for principal kudu/127.20.125.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.190.keytab.
I20260809 09:35:37.870579 20982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
/tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.125.190:43909
--webserver_interface=127.20.125.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.190.keytab
--principal=kudu/127.20.125.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.125.148:36641
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.125.190:43909
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 09:35:37.986948 11352 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 09:35:37.987231 11352 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 09:35:37.987293 11352 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 09:35:37.991281 11352 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260809 09:35:37.991329 11352 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260809 09:35:37.991344 11352 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260809 09:35:37.991358 11352 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 09:35:37.991376 11352 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 09:35:37.991444 11352 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 09:35:37.991467 11352 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 09:35:37.991495 11352 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 09:35:37.996747 11352 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.125.148:36641
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.20.125.190:43909
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.190.keytab
--principal=kudu/127.20.125.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.125.190:43909
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.125.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11352
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 08:38:55 UTC on ccycloud.quasar-njjleu.root.comops.site
build id 4575935
I20260809 09:35:37.997843 11352 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 09:35:37.999258 11352 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 09:35:38.008934 11360 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:35:38.009027 11362 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:35:38.009001 11352 server_base.cc:1034] running on GCE node
W20260809 09:35:38.009227 11359 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:35:38.009984 11352 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 09:35:38.011278 11352 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 09:35:38.012513 11352 hybrid_clock.cc:648] HybridClock initialized: now 1786268138012474 us; error 57 us; skew 500 ppm
Aug 09 09:35:38 dist-test-slave-7kzw krb5kdc[11335](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268138, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 09:35:38.015944 11352 init.cc:377] Logged in from keytab as kudu/127.20.125.190@KRBTEST.COM (short username kudu)
I20260809 09:35:38.016898 11352 webserver.cc:460] Webserver started at http://127.20.125.190:41935/ using document root <none> and password file <none>
I20260809 09:35:38.017498 11352 fs_manager.cc:359] Metadata directory not provided
I20260809 09:35:38.017601 11352 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 09:35:38.017855 11352 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 09:35:38.020059 11352 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/minicluster-data/master-0/data/instance:
uuid: "32d6a58804784035826482602ca4a412"
format_stamp: "Formatted at 2026-08-09 09:35:38 on dist-test-slave-7kzw"
server_key: "924dd932b65860656a17b47c1f2507df"
server_key_iv: "ed1d617e6d6196eee8d2a571900da52a"
server_key_version: "encryptionkey@0"
I20260809 09:35:38.020675 11352 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/minicluster-data/master-0/wal/instance:
uuid: "32d6a58804784035826482602ca4a412"
format_stamp: "Formatted at 2026-08-09 09:35:38 on dist-test-slave-7kzw"
server_key: "924dd932b65860656a17b47c1f2507df"
server_key_iv: "ed1d617e6d6196eee8d2a571900da52a"
server_key_version: "encryptionkey@0"
I20260809 09:35:38.024652 11352 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 09:35:38.027252 11373 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 09:35:38.028512 11352 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 09:35:38.028702 11352 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/minicluster-data/master-0/data,/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/minicluster-data/master-0/wal
uuid: "32d6a58804784035826482602ca4a412"
format_stamp: "Formatted at 2026-08-09 09:35:38 on dist-test-slave-7kzw"
server_key: "924dd932b65860656a17b47c1f2507df"
server_key_iv: "ed1d617e6d6196eee8d2a571900da52a"
server_key_version: "encryptionkey@0"
I20260809 09:35:38.028822 11352 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 09:35:38.048837 11352 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 09:35:38.052090 11352 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 09:35:38.052300 11352 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 09:35:38.059916 11352 rpc_server.cc:307] RPC server started. Bound to: 127.20.125.190:43909
I20260809 09:35:38.060249 11436 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.125.190:43909 every 8 connection(s)
I20260809 09:35:38.060971 11352 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/minicluster-data/master-0/data/info.pb
I20260809 09:35:38.063848 11438 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 09:35:38.068842 20982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu as pid 11352
I20260809 09:35:38.069039 20982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/minicluster-data/master-0/wal/instance
I20260809 09:35:38.069422 20982 external_mini_cluster.cc:1428] Setting key b867f3189c724a4f403d9e56350f2df5
I20260809 09:35:38.071151 11438 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 32d6a58804784035826482602ca4a412: Bootstrap starting.
I20260809 09:35:38.074328 11438 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 32d6a58804784035826482602ca4a412: Neither blocks nor log segments found. Creating new log.
I20260809 09:35:38.075371 11438 log.cc:824] T 00000000000000000000000000000000 P 32d6a58804784035826482602ca4a412: Log is configured to *not* fsync() on all Append() calls
I20260809 09:35:38.077713 11438 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 32d6a58804784035826482602ca4a412: No bootstrap required, opened a new log
Aug 09 09:35:38 dist-test-slave-7kzw krb5kdc[11335](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268137, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.125.190@KRBTEST.COM
I20260809 09:35:38.082751 11438 raft_consensus.cc:348] T 00000000000000000000000000000000 P 32d6a58804784035826482602ca4a412 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32d6a58804784035826482602ca4a412" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 43909 } }
I20260809 09:35:38.083061 11438 raft_consensus.cc:374] T 00000000000000000000000000000000 P 32d6a58804784035826482602ca4a412 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 09:35:38.083117 11438 raft_consensus.cc:729] T 00000000000000000000000000000000 P 32d6a58804784035826482602ca4a412 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32d6a58804784035826482602ca4a412, State: Initialized, Role: FOLLOWER
I20260809 09:35:38.083580 11438 consensus_queue.cc:260] T 00000000000000000000000000000000 P 32d6a58804784035826482602ca4a412 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32d6a58804784035826482602ca4a412" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 43909 } }
I20260809 09:35:38.083762 11438 raft_consensus.cc:388] T 00000000000000000000000000000000 P 32d6a58804784035826482602ca4a412 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 09:35:38.083863 11438 raft_consensus.cc:482] T 00000000000000000000000000000000 P 32d6a58804784035826482602ca4a412 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 09:35:38.083967 11438 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 32d6a58804784035826482602ca4a412 [term 0 FOLLOWER]: Advancing to term 1
I20260809 09:35:38.085242 11438 raft_consensus.cc:504] T 00000000000000000000000000000000 P 32d6a58804784035826482602ca4a412 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32d6a58804784035826482602ca4a412" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 43909 } }
I20260809 09:35:38.085621 11438 leader_election.cc:302] T 00000000000000000000000000000000 P 32d6a58804784035826482602ca4a412 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 32d6a58804784035826482602ca4a412; no voters:
I20260809 09:35:38.085964 11438 leader_election.cc:288] T 00000000000000000000000000000000 P 32d6a58804784035826482602ca4a412 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 09:35:38.086093 11444 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 32d6a58804784035826482602ca4a412 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 09:35:38.086400 11444 raft_consensus.cc:686] T 00000000000000000000000000000000 P 32d6a58804784035826482602ca4a412 [term 1 LEADER]: Becoming Leader. State: Replica: 32d6a58804784035826482602ca4a412, State: Running, Role: LEADER
I20260809 09:35:38.086611 11442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:35:38.071018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56582 (local address 127.20.125.190:43909)
0809 09:35:38.071507 (+ 489us) server_negotiation.cc:206] Beginning negotiation
0809 09:35:38.071530 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0809 09:35:38.071580 (+ 50us) server_negotiation.cc:407] Connection header received
0809 09:35:38.072592 (+ 1012us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:35:38.072618 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:35:38.073078 (+ 460us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:35:38.073474 (+ 396us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:35:38.074272 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:35:38.075946 (+ 1674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:35:38.076898 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:35:38.077682 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:35:38.077884 (+ 202us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:35:38.080845 (+ 2961us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:35:38.080854 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:35:38.080877 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:35:38.080925 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:35:38.083206 (+ 2281us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:35:38.083740 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:35:38.083746 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:35:38.083751 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:35:38.083932 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:35:38.084264 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:35:38.084267 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:35:38.084270 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:35:38.084783 (+ 513us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:35:38.084996 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0809 09:35:38.085244 (+ 248us) server_negotiation.cc:299] Negotiation successful
0809 09:35:38.085554 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":324,"thread_start_us":154,"threads_started":1}
I20260809 09:35:38.086797 11444 consensus_queue.cc:237] T 00000000000000000000000000000000 P 32d6a58804784035826482602ca4a412 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32d6a58804784035826482602ca4a412" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 43909 } }
I20260809 09:35:38.087291 11438 sys_catalog.cc:564] T 00000000000000000000000000000000 P 32d6a58804784035826482602ca4a412 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 09:35:38.088096 11444 sys_catalog.cc:455] T 00000000000000000000000000000000 P 32d6a58804784035826482602ca4a412 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 32d6a58804784035826482602ca4a412. Latest consensus state: current_term: 1 leader_uuid: "32d6a58804784035826482602ca4a412" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32d6a58804784035826482602ca4a412" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 43909 } } }
I20260809 09:35:38.088212 11444 sys_catalog.cc:458] T 00000000000000000000000000000000 P 32d6a58804784035826482602ca4a412 [sys.catalog]: This master's current role is: LEADER
I20260809 09:35:38.089094 11446 sys_catalog.cc:455] T 00000000000000000000000000000000 P 32d6a58804784035826482602ca4a412 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "32d6a58804784035826482602ca4a412" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32d6a58804784035826482602ca4a412" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 43909 } } }
I20260809 09:35:38.089290 11446 sys_catalog.cc:458] T 00000000000000000000000000000000 P 32d6a58804784035826482602ca4a412 [sys.catalog]: This master's current role is: LEADER
W20260809 09:35:38.092360 11466 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 32d6a58804784035826482602ca4a412: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 09:35:38.092458 11466 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 09:35:38.092561 11458 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 09:35:38.093329 11458 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 09:35:38.099465 11458 catalog_manager.cc:1347] Generated new cluster ID: d57ed51076194ef89d32e3ac81228f04
I20260809 09:35:38.099550 11458 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 09:35:38.110525 11458 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 09:35:38.111790 11458 catalog_manager.cc:1504] Loading token signing keys...
I20260809 09:35:38.120635 11458 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 32d6a58804784035826482602ca4a412: Generated new TSK 0
I20260809 09:35:38.121439 11458 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.20.125.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.125.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.125.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.129.keytab.
Entry for principal kudu/127.20.125.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.129.keytab.
I20260809 09:35:38.165843 20982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
/tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.125.129:0
--local_ip_for_outbound_sockets=127.20.125.129
--webserver_interface=127.20.125.129
--webserver_port=0
--tserver_master_addrs=127.20.125.190:43909
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.129.keytab
--principal=kudu/127.20.125.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.125.148:36641
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 09:35:38.283104 11479 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 09:35:38.283404 11479 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 09:35:38.283481 11479 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 09:35:38.287617 11479 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 09:35:38.287698 11479 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 09:35:38.287784 11479 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.125.129
I20260809 09:35:38.293309 11479 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.125.148:36641
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.129.keytab
--principal=kudu/127.20.125.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.125.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.125.129
--webserver_port=0
--tserver_master_addrs=127.20.125.190:43909
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11479
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.125.129
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 08:38:55 UTC on ccycloud.quasar-njjleu.root.comops.site
build id 4575935
I20260809 09:35:38.294390 11479 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 09:35:38.295481 11479 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 09:35:38.305384 11491 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:35:38.305390 11485 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:35:38.305496 11493 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:35:38.305689 11479 server_base.cc:1034] running on GCE node
I20260809 09:35:38.306170 11479 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 09:35:38.306849 11479 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 09:35:38.308056 11479 hybrid_clock.cc:648] HybridClock initialized: now 1786268138308037 us; error 30 us; skew 500 ppm
Aug 09 09:35:38 dist-test-slave-7kzw krb5kdc[11335](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268138, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 09:35:38.311051 11479 init.cc:377] Logged in from keytab as kudu/127.20.125.129@KRBTEST.COM (short username kudu)
I20260809 09:35:38.311972 11479 webserver.cc:460] Webserver started at http://127.20.125.129:35685/ using document root <none> and password file <none>
I20260809 09:35:38.312562 11479 fs_manager.cc:359] Metadata directory not provided
I20260809 09:35:38.312659 11479 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 09:35:38.312901 11479 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 09:35:38.315033 11479 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/minicluster-data/ts-0/data/instance:
uuid: "5ac1b284ab624af9bcc0235ebb0d2ead"
format_stamp: "Formatted at 2026-08-09 09:35:38 on dist-test-slave-7kzw"
server_key: "60cd82aebd07ab3a9b3bdf5d1ee88114"
server_key_iv: "cc27a2a6d8550174c0980958f956bb28"
server_key_version: "encryptionkey@0"
I20260809 09:35:38.315604 11479 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/minicluster-data/ts-0/wal/instance:
uuid: "5ac1b284ab624af9bcc0235ebb0d2ead"
format_stamp: "Formatted at 2026-08-09 09:35:38 on dist-test-slave-7kzw"
server_key: "60cd82aebd07ab3a9b3bdf5d1ee88114"
server_key_iv: "cc27a2a6d8550174c0980958f956bb28"
server_key_version: "encryptionkey@0"
I20260809 09:35:38.319509 11479 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260809 09:35:38.321889 11502 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 09:35:38.323156 11479 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 09:35:38.323297 11479 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/minicluster-data/ts-0/data,/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/minicluster-data/ts-0/wal
uuid: "5ac1b284ab624af9bcc0235ebb0d2ead"
format_stamp: "Formatted at 2026-08-09 09:35:38 on dist-test-slave-7kzw"
server_key: "60cd82aebd07ab3a9b3bdf5d1ee88114"
server_key_iv: "cc27a2a6d8550174c0980958f956bb28"
server_key_version: "encryptionkey@0"
I20260809 09:35:38.323414 11479 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 09:35:38.342136 11479 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 09:35:38.345497 11479 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 09:35:38.345713 11479 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 09:35:38.346312 11479 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 09:35:38.347158 11479 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 09:35:38.347231 11479 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 09:35:38.347321 11479 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 09:35:38.347368 11479 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 09:35:38.356539 11479 rpc_server.cc:307] RPC server started. Bound to: 127.20.125.129:42417
I20260809 09:35:38.356575 11673 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.125.129:42417 every 8 connection(s)
I20260809 09:35:38.357409 11479 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/minicluster-data/ts-0/data/info.pb
I20260809 09:35:38.363329 20982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu as pid 11479
I20260809 09:35:38.363445 20982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/minicluster-data/ts-0/wal/instance
I20260809 09:35:38.363822 20982 external_mini_cluster.cc:1428] Setting key 4ae7a884972d8110b111f57734c2ab3e
Aug 09 09:35:38 dist-test-slave-7kzw krb5kdc[11335](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268138, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.129@KRBTEST.COM for kudu/127.20.125.190@KRBTEST.COM
I20260809 09:35:38.371871 11442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:35:38.359218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.129:53013 (local address 127.20.125.190:43909)
0809 09:35:38.359374 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0809 09:35:38.359379 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 09:35:38.360226 (+ 847us) server_negotiation.cc:407] Connection header received
0809 09:35:38.361442 (+ 1216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:35:38.361445 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:35:38.361492 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:35:38.361571 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:35:38.362773 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:35:38.363335 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:35:38.364895 (+ 1560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:35:38.365195 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:35:38.365305 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:35:38.368058 (+ 2753us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:35:38.368063 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:35:38.368068 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:35:38.368100 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:35:38.369568 (+ 1468us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:35:38.370213 (+ 645us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:35:38.370216 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:35:38.370217 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:35:38.370295 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:35:38.370765 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:35:38.370767 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:35:38.370769 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:35:38.370956 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:35:38.371066 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0809 09:35:38.371551 (+ 485us) server_negotiation.cc:299] Negotiation successful
0809 09:35:38.371689 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 09:35:38.372706 11677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:35:38.359634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:43909 (local address 127.20.125.129:53013)
0809 09:35:38.360068 (+ 434us) negotiation.cc:107] Waiting for socket to connect
0809 09:35:38.360111 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 09:35:38.361227 (+ 1116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:35:38.361763 (+ 536us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:35:38.361769 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:35:38.362100 (+ 331us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:35:38.362627 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:35:38.362642 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:35:38.363616 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:35:38.363620 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:35:38.364779 (+ 1159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:35:38.364789 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:35:38.365357 (+ 568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:35:38.365363 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:35:38.365506 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:35:38.366109 (+ 603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 09:35:38.366125 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:35:38.367892 (+ 1767us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 09:35:38.369700 (+ 1808us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:35:38.369709 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:35:38.369725 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:35:38.370099 (+ 374us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:35:38.370393 (+ 294us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:35:38.370399 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:35:38.370404 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:35:38.370660 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:35:38.371068 (+ 408us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 09:35:38.371076 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 09:35:38.371311 (+ 235us) client_negotiation.cc:769] Sending connection context
0809 09:35:38.371492 (+ 181us) client_negotiation.cc:240] Negotiation successful
0809 09:35:38.371802 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":106,"threads_started":1}
I20260809 09:35:38.373886 11674 heartbeater.cc:347] Connected to a master server at 127.20.125.190:43909
I20260809 09:35:38.374128 11674 heartbeater.cc:464] Registering TS with master...
I20260809 09:35:38.374691 11674 heartbeater.cc:511] Master 127.20.125.190:43909 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.20.125.130@KRBTEST.COM; defaulting to no policy
I20260809 09:35:38.376268 11391 ts_manager.cc:194] Registered new tserver with Master: 5ac1b284ab624af9bcc0235ebb0d2ead (127.20.125.129:42417)
I20260809 09:35:38.377543 11391 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.125.129@KRBTEST.COM'} at 127.20.125.129:53013
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.125.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.125.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.130.keytab.
Entry for principal kudu/127.20.125.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.130.keytab.
I20260809 09:35:38.395730 20982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
/tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.125.130:0
--local_ip_for_outbound_sockets=127.20.125.130
--webserver_interface=127.20.125.130
--webserver_port=0
--tserver_master_addrs=127.20.125.190:43909
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.130.keytab
--principal=kudu/127.20.125.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.125.148:36641
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 09:35:38.511725 11683 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 09:35:38.511991 11683 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 09:35:38.512035 11683 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 09:35:38.516031 11683 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 09:35:38.516094 11683 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 09:35:38.516152 11683 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.125.130
I20260809 09:35:38.521396 11683 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.125.148:36641
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.130.keytab
--principal=kudu/127.20.125.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.125.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.125.130
--webserver_port=0
--tserver_master_addrs=127.20.125.190:43909
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11683
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.125.130
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 08:38:55 UTC on ccycloud.quasar-njjleu.root.comops.site
build id 4575935
I20260809 09:35:38.522531 11683 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 09:35:38.523514 11683 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 09:35:38.533561 11683 server_base.cc:1034] running on GCE node
W20260809 09:35:38.533736 11695 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:35:38.533634 11691 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:35:38.533640 11689 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:35:38.534250 11683 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 09:35:38.534948 11683 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 09:35:38.536136 11683 hybrid_clock.cc:648] HybridClock initialized: now 1786268138536098 us; error 51 us; skew 500 ppm
Aug 09 09:35:38 dist-test-slave-7kzw krb5kdc[11335](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268138, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 09:35:38.539094 11683 init.cc:377] Logged in from keytab as kudu/127.20.125.130@KRBTEST.COM (short username kudu)
I20260809 09:35:38.540005 11683 webserver.cc:460] Webserver started at http://127.20.125.130:40129/ using document root <none> and password file <none>
I20260809 09:35:38.540568 11683 fs_manager.cc:359] Metadata directory not provided
I20260809 09:35:38.540661 11683 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 09:35:38.540916 11683 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 09:35:38.543059 11683 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/minicluster-data/ts-1/data/instance:
uuid: "696cb7ca94004c299e1706b18b99498c"
format_stamp: "Formatted at 2026-08-09 09:35:38 on dist-test-slave-7kzw"
server_key: "e4ce44f94462a74aeb53e508794e8739"
server_key_iv: "98bda8f07cfb4c5d370c5a1017a92b61"
server_key_version: "encryptionkey@0"
I20260809 09:35:38.543622 11683 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/minicluster-data/ts-1/wal/instance:
uuid: "696cb7ca94004c299e1706b18b99498c"
format_stamp: "Formatted at 2026-08-09 09:35:38 on dist-test-slave-7kzw"
server_key: "e4ce44f94462a74aeb53e508794e8739"
server_key_iv: "98bda8f07cfb4c5d370c5a1017a92b61"
server_key_version: "encryptionkey@0"
I20260809 09:35:38.547322 11683 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260809 09:35:38.549700 11705 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 09:35:38.550971 11683 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260809 09:35:38.551105 11683 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/minicluster-data/ts-1/data,/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/minicluster-data/ts-1/wal
uuid: "696cb7ca94004c299e1706b18b99498c"
format_stamp: "Formatted at 2026-08-09 09:35:38 on dist-test-slave-7kzw"
server_key: "e4ce44f94462a74aeb53e508794e8739"
server_key_iv: "98bda8f07cfb4c5d370c5a1017a92b61"
server_key_version: "encryptionkey@0"
I20260809 09:35:38.551220 11683 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 09:35:38.569545 11683 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 09:35:38.572956 11683 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 09:35:38.573158 11683 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 09:35:38.573751 11683 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 09:35:38.574585 11683 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 09:35:38.574658 11683 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 09:35:38.574751 11683 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 09:35:38.574802 11683 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 09:35:38.583882 11683 rpc_server.cc:307] RPC server started. Bound to: 127.20.125.130:33021
I20260809 09:35:38.583909 11869 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.125.130:33021 every 8 connection(s)
I20260809 09:35:38.584777 11683 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/minicluster-data/ts-1/data/info.pb
I20260809 09:35:38.593148 20982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu as pid 11683
I20260809 09:35:38.593304 20982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/minicluster-data/ts-1/wal/instance
I20260809 09:35:38.593640 20982 external_mini_cluster.cc:1428] Setting key cee46ed36e488d60c179cf225364ad13
Aug 09 09:35:38 dist-test-slave-7kzw krb5kdc[11335](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268138, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.130@KRBTEST.COM for kudu/127.20.125.190@KRBTEST.COM
I20260809 09:35:38.599856 11442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:35:38.586567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.130:41575 (local address 127.20.125.190:43909)
0809 09:35:38.586706 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0809 09:35:38.586710 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 09:35:38.587598 (+ 888us) server_negotiation.cc:407] Connection header received
0809 09:35:38.589068 (+ 1470us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:35:38.589071 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:35:38.589132 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:35:38.589245 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:35:38.590572 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:35:38.591111 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:35:38.592258 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:35:38.592484 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:35:38.592593 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:35:38.595551 (+ 2958us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:35:38.595558 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:35:38.595564 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:35:38.595597 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:35:38.597312 (+ 1715us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:35:38.598179 (+ 867us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:35:38.598182 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:35:38.598183 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:35:38.598274 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:35:38.598745 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:35:38.598748 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:35:38.598749 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:35:38.598923 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:35:38.598999 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 09:35:38.599582 (+ 583us) server_negotiation.cc:299] Negotiation successful
0809 09:35:38.599738 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260809 09:35:38.600714 11872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:35:38.586984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:43909 (local address 127.20.125.130:41575)
0809 09:35:38.587450 (+ 466us) negotiation.cc:107] Waiting for socket to connect
0809 09:35:38.587492 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 09:35:38.588796 (+ 1304us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:35:38.589421 (+ 625us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:35:38.589427 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:35:38.589881 (+ 454us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:35:38.590382 (+ 501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:35:38.590396 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:35:38.591245 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:35:38.591248 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:35:38.592124 (+ 876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:35:38.592133 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:35:38.592646 (+ 513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:35:38.592650 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:35:38.592819 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:35:38.593559 (+ 740us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 09:35:38.593576 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:35:38.595387 (+ 1811us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 09:35:38.597565 (+ 2178us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:35:38.597584 (+ 19us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:35:38.597602 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:35:38.598052 (+ 450us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:35:38.598370 (+ 318us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:35:38.598374 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:35:38.598378 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:35:38.598624 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:35:38.599030 (+ 406us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 09:35:38.599038 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 09:35:38.599358 (+ 320us) client_negotiation.cc:769] Sending connection context
0809 09:35:38.599519 (+ 161us) client_negotiation.cc:240] Negotiation successful
0809 09:35:38.599784 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":282,"thread_start_us":119,"threads_started":1}
I20260809 09:35:38.602083 11870 heartbeater.cc:347] Connected to a master server at 127.20.125.190:43909
I20260809 09:35:38.602345 11870 heartbeater.cc:464] Registering TS with master...
I20260809 09:35:38.602908 11870 heartbeater.cc:511] Master 127.20.125.190:43909 requested a full tablet report, sending...
I20260809 09:35:38.604188 11391 ts_manager.cc:194] Registered new tserver with Master: 696cb7ca94004c299e1706b18b99498c (127.20.125.130:33021)
I20260809 09:35:38.604971 11391 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.125.130@KRBTEST.COM'} at 127.20.125.130:41575
WARNING: no policy specified for kudu/127.20.125.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.125.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.125.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.131.keytab.
Entry for principal kudu/127.20.125.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.131.keytab.
I20260809 09:35:38.629227 20982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
/tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.125.131:0
--local_ip_for_outbound_sockets=127.20.125.131
--webserver_interface=127.20.125.131
--webserver_port=0
--tserver_master_addrs=127.20.125.190:43909
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.131.keytab
--principal=kudu/127.20.125.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.125.148:36641
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 09:35:38.744827 11878 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 09:35:38.745123 11878 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 09:35:38.745208 11878 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 09:35:38.749209 11878 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 09:35:38.749302 11878 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 09:35:38.749398 11878 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.125.131
I20260809 09:35:38.754688 11878 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.125.148:36641
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.131.keytab
--principal=kudu/127.20.125.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.125.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.125.131
--webserver_port=0
--tserver_master_addrs=127.20.125.190:43909
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11878
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.125.131
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 08:38:55 UTC on ccycloud.quasar-njjleu.root.comops.site
build id 4575935
I20260809 09:35:38.755828 11878 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 09:35:38.756858 11878 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 09:35:38.766922 11888 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:35:38.766948 11885 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:35:38.766942 11884 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:35:38.767014 11878 server_base.cc:1034] running on GCE node
I20260809 09:35:38.767594 11878 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 09:35:38.768286 11878 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 09:35:38.769469 11878 hybrid_clock.cc:648] HybridClock initialized: now 1786268138769432 us; error 47 us; skew 500 ppm
Aug 09 09:35:38 dist-test-slave-7kzw krb5kdc[11335](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268138, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 09:35:38.772449 11878 init.cc:377] Logged in from keytab as kudu/127.20.125.131@KRBTEST.COM (short username kudu)
I20260809 09:35:38.773403 11878 webserver.cc:460] Webserver started at http://127.20.125.131:38713/ using document root <none> and password file <none>
I20260809 09:35:38.773975 11878 fs_manager.cc:359] Metadata directory not provided
I20260809 09:35:38.774040 11878 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 09:35:38.774331 11878 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 09:35:38.776664 11878 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/minicluster-data/ts-2/data/instance:
uuid: "e326d8d7643f49fba2572694f4a9a3c3"
format_stamp: "Formatted at 2026-08-09 09:35:38 on dist-test-slave-7kzw"
server_key: "b85b39b0443c81ffef29c4c085c8d970"
server_key_iv: "74e02f4580a8d8c897cee5a7de119270"
server_key_version: "encryptionkey@0"
I20260809 09:35:38.777284 11878 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/minicluster-data/ts-2/wal/instance:
uuid: "e326d8d7643f49fba2572694f4a9a3c3"
format_stamp: "Formatted at 2026-08-09 09:35:38 on dist-test-slave-7kzw"
server_key: "b85b39b0443c81ffef29c4c085c8d970"
server_key_iv: "74e02f4580a8d8c897cee5a7de119270"
server_key_version: "encryptionkey@0"
I20260809 09:35:38.781172 11878 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260809 09:35:38.783732 11897 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 09:35:38.785158 11878 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 09:35:38.785356 11878 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/minicluster-data/ts-2/data,/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/minicluster-data/ts-2/wal
uuid: "e326d8d7643f49fba2572694f4a9a3c3"
format_stamp: "Formatted at 2026-08-09 09:35:38 on dist-test-slave-7kzw"
server_key: "b85b39b0443c81ffef29c4c085c8d970"
server_key_iv: "74e02f4580a8d8c897cee5a7de119270"
server_key_version: "encryptionkey@0"
I20260809 09:35:38.785493 11878 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 09:35:38.807202 11878 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 09:35:38.810717 11878 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 09:35:38.810935 11878 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 09:35:38.811591 11878 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 09:35:38.812474 11878 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 09:35:38.812546 11878 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 09:35:38.812635 11878 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 09:35:38.812662 11878 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 09:35:38.822515 11878 rpc_server.cc:307] RPC server started. Bound to: 127.20.125.131:46207
I20260809 09:35:38.822520 12059 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.125.131:46207 every 8 connection(s)
I20260809 09:35:38.823491 11878 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/minicluster-data/ts-2/data/info.pb
I20260809 09:35:38.827075 20982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu as pid 11878
I20260809 09:35:38.827358 20982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/minicluster-data/ts-2/wal/instance
I20260809 09:35:38.827693 20982 external_mini_cluster.cc:1428] Setting key 9271139a6e16abd5c503eeeaafe2f35a
Aug 09 09:35:38 dist-test-slave-7kzw krb5kdc[11335](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268138, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.131@KRBTEST.COM for kudu/127.20.125.190@KRBTEST.COM
I20260809 09:35:38.839731 11442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:35:38.825470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.131:50895 (local address 127.20.125.190:43909)
0809 09:35:38.825673 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0809 09:35:38.825678 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 09:35:38.826762 (+ 1084us) server_negotiation.cc:407] Connection header received
0809 09:35:38.828772 (+ 2010us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:35:38.828782 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:35:38.828831 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:35:38.828900 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:35:38.830578 (+ 1678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:35:38.831126 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:35:38.832710 (+ 1584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:35:38.832994 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:35:38.833088 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:35:38.835949 (+ 2861us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:35:38.835953 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:35:38.835955 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:35:38.835983 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:35:38.837506 (+ 1523us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:35:38.838053 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:35:38.838056 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:35:38.838058 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:35:38.838145 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:35:38.838602 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:35:38.838606 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:35:38.838607 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:35:38.838832 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:35:38.838952 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0809 09:35:38.839488 (+ 536us) server_negotiation.cc:299] Negotiation successful
0809 09:35:38.839602 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100}
I20260809 09:35:38.840674 12063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:35:38.825887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:43909 (local address 127.20.125.131:50895)
0809 09:35:38.826550 (+ 663us) negotiation.cc:107] Waiting for socket to connect
0809 09:35:38.826626 (+ 76us) client_negotiation.cc:174] Beginning negotiation
0809 09:35:38.828514 (+ 1888us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:35:38.829152 (+ 638us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:35:38.829163 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:35:38.829638 (+ 475us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:35:38.830363 (+ 725us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:35:38.830381 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:35:38.831275 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:35:38.831278 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:35:38.832572 (+ 1294us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:35:38.832583 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:35:38.833251 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:35:38.833256 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:35:38.833475 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:35:38.834182 (+ 707us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 09:35:38.834199 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:35:38.835779 (+ 1580us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 09:35:38.837627 (+ 1848us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:35:38.837632 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:35:38.837645 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:35:38.837917 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:35:38.838240 (+ 323us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:35:38.838242 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:35:38.838244 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:35:38.838386 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:35:38.838954 (+ 568us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 09:35:38.838960 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 09:35:38.839211 (+ 251us) client_negotiation.cc:769] Sending connection context
0809 09:35:38.839375 (+ 164us) client_negotiation.cc:240] Negotiation successful
0809 09:35:38.839638 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":372,"thread_start_us":156,"threads_started":1}
I20260809 09:35:38.841822 12061 heartbeater.cc:347] Connected to a master server at 127.20.125.190:43909
I20260809 09:35:38.842078 12061 heartbeater.cc:464] Registering TS with master...
I20260809 09:35:38.842639 12061 heartbeater.cc:511] Master 127.20.125.190:43909 requested a full tablet report, sending...
I20260809 09:35:38.843750 11391 ts_manager.cc:194] Registered new tserver with Master: e326d8d7643f49fba2572694f4a9a3c3 (127.20.125.131:46207)
I20260809 09:35:38.844383 11391 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.125.131@KRBTEST.COM'} at 127.20.125.131:50895
I20260809 09:35:38.853914 20982 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 09:35:38.864830 11442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:35:38.856300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56598 (local address 127.20.125.190:43909)
0809 09:35:38.856514 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0809 09:35:38.856520 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 09:35:38.856676 (+ 156us) server_negotiation.cc:407] Connection header received
0809 09:35:38.856901 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:35:38.856905 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:35:38.856960 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:35:38.857054 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:35:38.857859 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:35:38.858564 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:35:38.859496 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:35:38.859754 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:35:38.859872 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:35:38.861412 (+ 1540us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:35:38.861416 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:35:38.861419 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:35:38.861454 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:35:38.862951 (+ 1497us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:35:38.863420 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:35:38.863424 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:35:38.863426 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:35:38.863520 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:35:38.863828 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:35:38.863831 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:35:38.863833 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:35:38.864083 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:35:38.864266 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0809 09:35:38.864509 (+ 243us) server_negotiation.cc:299] Negotiation successful
0809 09:35:38.864643 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104}
I20260809 09:35:38.868108 11391 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56598:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 09:35:38.871547 11391 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 09:35:38.888332 12076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:35:38.881446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.129:42417 (local address 127.0.0.1:37568)
0809 09:35:38.882642 (+ 1196us) negotiation.cc:107] Waiting for socket to connect
0809 09:35:38.882658 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 09:35:38.882756 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:35:38.883483 (+ 727us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:35:38.883486 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:35:38.883504 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 09:35:38.883618 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:35:38.883625 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:35:38.885164 (+ 1539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:35:38.885167 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:35:38.886705 (+ 1538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:35:38.886716 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:35:38.887755 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:35:38.887759 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:35:38.887865 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:35:38.887902 (+ 37us) client_negotiation.cc:769] Sending connection context
0809 09:35:38.888014 (+ 112us) client_negotiation.cc:240] Negotiation successful
0809 09:35:38.888140 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1126,"mutex_wait_us":77,"thread_start_us":130,"threads_started":1}
I20260809 09:35:38.888814 12075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:35:38.881465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52746 (local address 127.20.125.131:46207)
0809 09:35:38.882415 (+ 950us) server_negotiation.cc:206] Beginning negotiation
0809 09:35:38.882429 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 09:35:38.882457 (+ 28us) server_negotiation.cc:407] Connection header received
0809 09:35:38.882587 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:35:38.882595 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:35:38.882850 (+ 255us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:35:38.883018 (+ 168us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 09:35:38.883598 (+ 580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:35:38.884993 (+ 1395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:35:38.886772 (+ 1779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:35:38.887617 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:35:38.887779 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:35:38.887920 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0809 09:35:38.888443 (+ 523us) server_negotiation.cc:299] Negotiation successful
0809 09:35:38.888624 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":833,"thread_start_us":134,"threads_started":1}
I20260809 09:35:38.888846 12073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:35:38.881324 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.131:46207 (local address 127.0.0.1:52746)
0809 09:35:38.881692 (+ 368us) negotiation.cc:107] Waiting for socket to connect
0809 09:35:38.881740 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0809 09:35:38.881872 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:35:38.883233 (+ 1361us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:35:38.883240 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:35:38.883267 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 09:35:38.883443 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:35:38.883455 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:35:38.885153 (+ 1698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:35:38.885158 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:35:38.886637 (+ 1479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:35:38.886649 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:35:38.887998 (+ 1349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:35:38.888002 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:35:38.888102 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:35:38.888130 (+ 28us) client_negotiation.cc:769] Sending connection context
0809 09:35:38.888218 (+ 88us) client_negotiation.cc:240] Negotiation successful
0809 09:35:38.888310 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":194,"mutex_wait_us":150,"thread_start_us":113,"threads_started":1}
I20260809 09:35:38.889073 12074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:35:38.882208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.130:33021 (local address 127.0.0.1:34698)
0809 09:35:38.883070 (+ 862us) negotiation.cc:107] Waiting for socket to connect
0809 09:35:38.883082 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 09:35:38.883210 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:35:38.883841 (+ 631us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:35:38.883843 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:35:38.883857 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 09:35:38.883926 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:35:38.883930 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:35:38.884977 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:35:38.884984 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:35:38.887083 (+ 2099us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:35:38.887092 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:35:38.888566 (+ 1474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:35:38.888570 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:35:38.888702 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:35:38.888729 (+ 27us) client_negotiation.cc:769] Sending connection context
0809 09:35:38.888809 (+ 80us) client_negotiation.cc:240] Negotiation successful
0809 09:35:38.888895 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":789,"mutex_wait_us":831,"thread_start_us":99,"threads_started":1}
I20260809 09:35:38.889508 12082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:35:38.882676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34698 (local address 127.20.125.130:33021)
0809 09:35:38.883032 (+ 356us) server_negotiation.cc:206] Beginning negotiation
0809 09:35:38.883044 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 09:35:38.883183 (+ 139us) server_negotiation.cc:407] Connection header received
0809 09:35:38.883326 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:35:38.883335 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:35:38.883514 (+ 179us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:35:38.883663 (+ 149us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 09:35:38.884043 (+ 380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:35:38.884814 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:35:38.887227 (+ 2413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:35:38.888196 (+ 969us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:35:38.888318 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:35:38.888448 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0809 09:35:38.889261 (+ 813us) server_negotiation.cc:299] Negotiation successful
0809 09:35:38.889376 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":259,"thread_start_us":172,"threads_started":1}
I20260809 09:35:38.889550 12080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:35:38.882231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37568 (local address 127.20.125.129:42417)
0809 09:35:38.883014 (+ 783us) server_negotiation.cc:206] Beginning negotiation
0809 09:35:38.883021 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 09:35:38.883045 (+ 24us) server_negotiation.cc:407] Connection header received
0809 09:35:38.883128 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:35:38.883133 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:35:38.883340 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:35:38.883505 (+ 165us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 09:35:38.883741 (+ 236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:35:38.884993 (+ 1252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:35:38.886856 (+ 1863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:35:38.887634 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:35:38.887779 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:35:38.887919 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0809 09:35:38.889311 (+ 1392us) server_negotiation.cc:299] Negotiation successful
0809 09:35:38.889416 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":669,"thread_start_us":97,"threads_started":1}
I20260809 09:35:38.891853 11969 tablet_service.cc:1467] Processing CreateTablet for tablet ed6c7c608bf74e968e295585c3c0d1b3 (DEFAULT_TABLE table=test-table [id=7dea63d3198940879c82c8c5ceebe710]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 09:35:38.892060 11581 tablet_service.cc:1467] Processing CreateTablet for tablet ed6c7c608bf74e968e295585c3c0d1b3 (DEFAULT_TABLE table=test-table [id=7dea63d3198940879c82c8c5ceebe710]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 09:35:38.893051 11969 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ed6c7c608bf74e968e295585c3c0d1b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 09:35:38.893160 11581 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ed6c7c608bf74e968e295585c3c0d1b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 09:35:38.894613 11777 tablet_service.cc:1467] Processing CreateTablet for tablet ed6c7c608bf74e968e295585c3c0d1b3 (DEFAULT_TABLE table=test-table [id=7dea63d3198940879c82c8c5ceebe710]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 09:35:38.895742 11777 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ed6c7c608bf74e968e295585c3c0d1b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 09:35:38.899878 12083 tablet_bootstrap.cc:492] T ed6c7c608bf74e968e295585c3c0d1b3 P e326d8d7643f49fba2572694f4a9a3c3: Bootstrap starting.
I20260809 09:35:38.901669 12084 tablet_bootstrap.cc:492] T ed6c7c608bf74e968e295585c3c0d1b3 P 5ac1b284ab624af9bcc0235ebb0d2ead: Bootstrap starting.
I20260809 09:35:38.902627 12083 tablet_bootstrap.cc:654] T ed6c7c608bf74e968e295585c3c0d1b3 P e326d8d7643f49fba2572694f4a9a3c3: Neither blocks nor log segments found. Creating new log.
I20260809 09:35:38.903609 12083 log.cc:824] T ed6c7c608bf74e968e295585c3c0d1b3 P e326d8d7643f49fba2572694f4a9a3c3: Log is configured to *not* fsync() on all Append() calls
I20260809 09:35:38.904166 12085 tablet_bootstrap.cc:492] T ed6c7c608bf74e968e295585c3c0d1b3 P 696cb7ca94004c299e1706b18b99498c: Bootstrap starting.
I20260809 09:35:38.904196 12084 tablet_bootstrap.cc:654] T ed6c7c608bf74e968e295585c3c0d1b3 P 5ac1b284ab624af9bcc0235ebb0d2ead: Neither blocks nor log segments found. Creating new log.
I20260809 09:35:38.905046 12084 log.cc:824] T ed6c7c608bf74e968e295585c3c0d1b3 P 5ac1b284ab624af9bcc0235ebb0d2ead: Log is configured to *not* fsync() on all Append() calls
I20260809 09:35:38.905714 12083 tablet_bootstrap.cc:492] T ed6c7c608bf74e968e295585c3c0d1b3 P e326d8d7643f49fba2572694f4a9a3c3: No bootstrap required, opened a new log
I20260809 09:35:38.905923 12083 ts_tablet_manager.cc:1397] T ed6c7c608bf74e968e295585c3c0d1b3 P e326d8d7643f49fba2572694f4a9a3c3: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260809 09:35:38.906750 12085 tablet_bootstrap.cc:654] T ed6c7c608bf74e968e295585c3c0d1b3 P 696cb7ca94004c299e1706b18b99498c: Neither blocks nor log segments found. Creating new log.
I20260809 09:35:38.907300 12084 tablet_bootstrap.cc:492] T ed6c7c608bf74e968e295585c3c0d1b3 P 5ac1b284ab624af9bcc0235ebb0d2ead: No bootstrap required, opened a new log
I20260809 09:35:38.907500 12084 ts_tablet_manager.cc:1397] T ed6c7c608bf74e968e295585c3c0d1b3 P 5ac1b284ab624af9bcc0235ebb0d2ead: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260809 09:35:38.907641 12085 log.cc:824] T ed6c7c608bf74e968e295585c3c0d1b3 P 696cb7ca94004c299e1706b18b99498c: Log is configured to *not* fsync() on all Append() calls
I20260809 09:35:38.909848 12085 tablet_bootstrap.cc:492] T ed6c7c608bf74e968e295585c3c0d1b3 P 696cb7ca94004c299e1706b18b99498c: No bootstrap required, opened a new log
I20260809 09:35:38.909665 12083 raft_consensus.cc:348] T ed6c7c608bf74e968e295585c3c0d1b3 P e326d8d7643f49fba2572694f4a9a3c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e326d8d7643f49fba2572694f4a9a3c3" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 46207 } } peers { permanent_uuid: "5ac1b284ab624af9bcc0235ebb0d2ead" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 42417 } } peers { permanent_uuid: "696cb7ca94004c299e1706b18b99498c" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 33021 } }
I20260809 09:35:38.910044 12085 ts_tablet_manager.cc:1397] T ed6c7c608bf74e968e295585c3c0d1b3 P 696cb7ca94004c299e1706b18b99498c: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260809 09:35:38.910068 12083 raft_consensus.cc:374] T ed6c7c608bf74e968e295585c3c0d1b3 P e326d8d7643f49fba2572694f4a9a3c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 09:35:38.910141 12083 raft_consensus.cc:729] T ed6c7c608bf74e968e295585c3c0d1b3 P e326d8d7643f49fba2572694f4a9a3c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e326d8d7643f49fba2572694f4a9a3c3, State: Initialized, Role: FOLLOWER
I20260809 09:35:38.910696 12083 consensus_queue.cc:260] T ed6c7c608bf74e968e295585c3c0d1b3 P e326d8d7643f49fba2572694f4a9a3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e326d8d7643f49fba2572694f4a9a3c3" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 46207 } } peers { permanent_uuid: "5ac1b284ab624af9bcc0235ebb0d2ead" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 42417 } } peers { permanent_uuid: "696cb7ca94004c299e1706b18b99498c" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 33021 } }
I20260809 09:35:38.911724 12083 ts_tablet_manager.cc:1428] T ed6c7c608bf74e968e295585c3c0d1b3 P e326d8d7643f49fba2572694f4a9a3c3: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260809 09:35:38.912127 12061 heartbeater.cc:503] Master 127.20.125.190:43909 was elected leader, sending a full tablet report...
I20260809 09:35:38.912037 12084 raft_consensus.cc:348] T ed6c7c608bf74e968e295585c3c0d1b3 P 5ac1b284ab624af9bcc0235ebb0d2ead [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e326d8d7643f49fba2572694f4a9a3c3" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 46207 } } peers { permanent_uuid: "5ac1b284ab624af9bcc0235ebb0d2ead" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 42417 } } peers { permanent_uuid: "696cb7ca94004c299e1706b18b99498c" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 33021 } }
I20260809 09:35:38.912292 12084 raft_consensus.cc:374] T ed6c7c608bf74e968e295585c3c0d1b3 P 5ac1b284ab624af9bcc0235ebb0d2ead [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 09:35:38.912351 12084 raft_consensus.cc:729] T ed6c7c608bf74e968e295585c3c0d1b3 P 5ac1b284ab624af9bcc0235ebb0d2ead [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ac1b284ab624af9bcc0235ebb0d2ead, State: Initialized, Role: FOLLOWER
I20260809 09:35:38.912803 12084 consensus_queue.cc:260] T ed6c7c608bf74e968e295585c3c0d1b3 P 5ac1b284ab624af9bcc0235ebb0d2ead [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e326d8d7643f49fba2572694f4a9a3c3" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 46207 } } peers { permanent_uuid: "5ac1b284ab624af9bcc0235ebb0d2ead" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 42417 } } peers { permanent_uuid: "696cb7ca94004c299e1706b18b99498c" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 33021 } }
I20260809 09:35:38.913511 11674 heartbeater.cc:503] Master 127.20.125.190:43909 was elected leader, sending a full tablet report...
I20260809 09:35:38.913818 12084 ts_tablet_manager.cc:1428] T ed6c7c608bf74e968e295585c3c0d1b3 P 5ac1b284ab624af9bcc0235ebb0d2ead: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260809 09:35:38.914047 12085 raft_consensus.cc:348] T ed6c7c608bf74e968e295585c3c0d1b3 P 696cb7ca94004c299e1706b18b99498c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e326d8d7643f49fba2572694f4a9a3c3" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 46207 } } peers { permanent_uuid: "5ac1b284ab624af9bcc0235ebb0d2ead" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 42417 } } peers { permanent_uuid: "696cb7ca94004c299e1706b18b99498c" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 33021 } }
I20260809 09:35:38.914273 12085 raft_consensus.cc:374] T ed6c7c608bf74e968e295585c3c0d1b3 P 696cb7ca94004c299e1706b18b99498c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 09:35:38.914305 12085 raft_consensus.cc:729] T ed6c7c608bf74e968e295585c3c0d1b3 P 696cb7ca94004c299e1706b18b99498c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 696cb7ca94004c299e1706b18b99498c, State: Initialized, Role: FOLLOWER
I20260809 09:35:38.914773 12085 consensus_queue.cc:260] T ed6c7c608bf74e968e295585c3c0d1b3 P 696cb7ca94004c299e1706b18b99498c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e326d8d7643f49fba2572694f4a9a3c3" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 46207 } } peers { permanent_uuid: "5ac1b284ab624af9bcc0235ebb0d2ead" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 42417 } } peers { permanent_uuid: "696cb7ca94004c299e1706b18b99498c" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 33021 } }
I20260809 09:35:38.915431 11870 heartbeater.cc:503] Master 127.20.125.190:43909 was elected leader, sending a full tablet report...
I20260809 09:35:38.915812 12085 ts_tablet_manager.cc:1428] T ed6c7c608bf74e968e295585c3c0d1b3 P 696cb7ca94004c299e1706b18b99498c: Time spent starting tablet: real 0.006s user 0.003s sys 0.004s
I20260809 09:35:38.917037 12093 raft_consensus.cc:482] T ed6c7c608bf74e968e295585c3c0d1b3 P 5ac1b284ab624af9bcc0235ebb0d2ead [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 09:35:38.917304 12093 raft_consensus.cc:504] T ed6c7c608bf74e968e295585c3c0d1b3 P 5ac1b284ab624af9bcc0235ebb0d2ead [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e326d8d7643f49fba2572694f4a9a3c3" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 46207 } } peers { permanent_uuid: "5ac1b284ab624af9bcc0235ebb0d2ead" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 42417 } } peers { permanent_uuid: "696cb7ca94004c299e1706b18b99498c" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 33021 } }
I20260809 09:35:38.919394 12093 leader_election.cc:288] T ed6c7c608bf74e968e295585c3c0d1b3 P 5ac1b284ab624af9bcc0235ebb0d2ead [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e326d8d7643f49fba2572694f4a9a3c3 (127.20.125.131:46207), 696cb7ca94004c299e1706b18b99498c (127.20.125.130:33021)
I20260809 09:35:38.923213 12082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:35:38.918825 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.129:35919 (local address 127.20.125.130:33021)
0809 09:35:38.918956 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0809 09:35:38.918961 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 09:35:38.919336 (+ 375us) server_negotiation.cc:407] Connection header received
0809 09:35:38.919549 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:35:38.919552 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:35:38.919604 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:35:38.919668 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 09:35:38.920127 (+ 459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:35:38.920632 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:35:38.922139 (+ 1507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:35:38.922629 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:35:38.922776 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:35:38.922831 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0809 09:35:38.923018 (+ 187us) server_negotiation.cc:299] Negotiation successful
0809 09:35:38.923087 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260809 09:35:38.923252 12095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:35:38.918672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.130:33021 (local address 127.20.125.129:35919)
0809 09:35:38.919251 (+ 579us) negotiation.cc:107] Waiting for socket to connect
0809 09:35:38.919268 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 09:35:38.919417 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:35:38.919816 (+ 399us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:35:38.919819 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:35:38.919837 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 09:35:38.919988 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:35:38.919997 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:35:38.920766 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:35:38.920769 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:35:38.922009 (+ 1240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:35:38.922020 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:35:38.922773 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:35:38.922777 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:35:38.922875 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:35:38.922900 (+ 25us) client_negotiation.cc:769] Sending connection context
0809 09:35:38.923002 (+ 102us) client_negotiation.cc:240] Negotiation successful
0809 09:35:38.923075 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":480,"mutex_wait_us":29,"thread_start_us":122,"threads_started":1}
I20260809 09:35:38.923962 12075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:35:38.918774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.129:49189 (local address 127.20.125.131:46207)
0809 09:35:38.919116 (+ 342us) server_negotiation.cc:206] Beginning negotiation
0809 09:35:38.919122 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 09:35:38.919560 (+ 438us) server_negotiation.cc:407] Connection header received
0809 09:35:38.919784 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:35:38.919787 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:35:38.919836 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:35:38.919913 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 09:35:38.920799 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:35:38.921539 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:35:38.922839 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:35:38.923264 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:35:38.923426 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:35:38.923471 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0809 09:35:38.923782 (+ 311us) server_negotiation.cc:299] Negotiation successful
0809 09:35:38.923857 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":266}
I20260809 09:35:38.923966 12096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:35:38.918673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.131:46207 (local address 127.20.125.129:49189)
0809 09:35:38.919408 (+ 735us) negotiation.cc:107] Waiting for socket to connect
0809 09:35:38.919423 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 09:35:38.919634 (+ 211us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:35:38.919934 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:35:38.919937 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:35:38.919953 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 09:35:38.920052 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:35:38.920056 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:35:38.921687 (+ 1631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:35:38.921690 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:35:38.922705 (+ 1015us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:35:38.922716 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:35:38.923531 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:35:38.923534 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:35:38.923633 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:35:38.923661 (+ 28us) client_negotiation.cc:769] Sending connection context
0809 09:35:38.923750 (+ 89us) client_negotiation.cc:240] Negotiation successful
0809 09:35:38.923823 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":666,"mutex_wait_us":218,"thread_start_us":106,"threads_started":1}
I20260809 09:35:38.923938 11803 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed6c7c608bf74e968e295585c3c0d1b3" candidate_uuid: "5ac1b284ab624af9bcc0235ebb0d2ead" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "696cb7ca94004c299e1706b18b99498c" is_pre_election: true
I20260809 09:35:38.924264 11803 raft_consensus.cc:2454] T ed6c7c608bf74e968e295585c3c0d1b3 P 696cb7ca94004c299e1706b18b99498c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5ac1b284ab624af9bcc0235ebb0d2ead in term 0.
I20260809 09:35:38.924582 11995 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed6c7c608bf74e968e295585c3c0d1b3" candidate_uuid: "5ac1b284ab624af9bcc0235ebb0d2ead" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e326d8d7643f49fba2572694f4a9a3c3" is_pre_election: true
I20260809 09:35:38.924808 11506 leader_election.cc:302] T ed6c7c608bf74e968e295585c3c0d1b3 P 5ac1b284ab624af9bcc0235ebb0d2ead [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5ac1b284ab624af9bcc0235ebb0d2ead, 696cb7ca94004c299e1706b18b99498c; no voters:
I20260809 09:35:38.924876 11995 raft_consensus.cc:2454] T ed6c7c608bf74e968e295585c3c0d1b3 P e326d8d7643f49fba2572694f4a9a3c3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5ac1b284ab624af9bcc0235ebb0d2ead in term 0.
I20260809 09:35:38.925133 12093 raft_consensus.cc:2781] T ed6c7c608bf74e968e295585c3c0d1b3 P 5ac1b284ab624af9bcc0235ebb0d2ead [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 09:35:38.925243 12093 raft_consensus.cc:482] T ed6c7c608bf74e968e295585c3c0d1b3 P 5ac1b284ab624af9bcc0235ebb0d2ead [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 09:35:38.925297 12093 raft_consensus.cc:3037] T ed6c7c608bf74e968e295585c3c0d1b3 P 5ac1b284ab624af9bcc0235ebb0d2ead [term 0 FOLLOWER]: Advancing to term 1
I20260809 09:35:38.926546 12093 raft_consensus.cc:504] T ed6c7c608bf74e968e295585c3c0d1b3 P 5ac1b284ab624af9bcc0235ebb0d2ead [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e326d8d7643f49fba2572694f4a9a3c3" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 46207 } } peers { permanent_uuid: "5ac1b284ab624af9bcc0235ebb0d2ead" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 42417 } } peers { permanent_uuid: "696cb7ca94004c299e1706b18b99498c" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 33021 } }
I20260809 09:35:38.926956 12093 leader_election.cc:288] T ed6c7c608bf74e968e295585c3c0d1b3 P 5ac1b284ab624af9bcc0235ebb0d2ead [CANDIDATE]: Term 1 election: Requested vote from peers e326d8d7643f49fba2572694f4a9a3c3 (127.20.125.131:46207), 696cb7ca94004c299e1706b18b99498c (127.20.125.130:33021)
I20260809 09:35:38.927258 11995 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed6c7c608bf74e968e295585c3c0d1b3" candidate_uuid: "5ac1b284ab624af9bcc0235ebb0d2ead" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e326d8d7643f49fba2572694f4a9a3c3"
I20260809 09:35:38.927380 11803 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed6c7c608bf74e968e295585c3c0d1b3" candidate_uuid: "5ac1b284ab624af9bcc0235ebb0d2ead" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "696cb7ca94004c299e1706b18b99498c"
I20260809 09:35:38.927428 11995 raft_consensus.cc:3037] T ed6c7c608bf74e968e295585c3c0d1b3 P e326d8d7643f49fba2572694f4a9a3c3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 09:35:38.927518 11803 raft_consensus.cc:3037] T ed6c7c608bf74e968e295585c3c0d1b3 P 696cb7ca94004c299e1706b18b99498c [term 0 FOLLOWER]: Advancing to term 1
I20260809 09:35:38.928604 11995 raft_consensus.cc:2454] T ed6c7c608bf74e968e295585c3c0d1b3 P e326d8d7643f49fba2572694f4a9a3c3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5ac1b284ab624af9bcc0235ebb0d2ead in term 1.
I20260809 09:35:38.928622 11803 raft_consensus.cc:2454] T ed6c7c608bf74e968e295585c3c0d1b3 P 696cb7ca94004c299e1706b18b99498c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5ac1b284ab624af9bcc0235ebb0d2ead in term 1.
I20260809 09:35:38.928946 11509 leader_election.cc:302] T ed6c7c608bf74e968e295585c3c0d1b3 P 5ac1b284ab624af9bcc0235ebb0d2ead [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5ac1b284ab624af9bcc0235ebb0d2ead, e326d8d7643f49fba2572694f4a9a3c3; no voters:
I20260809 09:35:38.929170 12093 raft_consensus.cc:2781] T ed6c7c608bf74e968e295585c3c0d1b3 P 5ac1b284ab624af9bcc0235ebb0d2ead [term 1 FOLLOWER]: Leader election won for term 1
I20260809 09:35:38.929407 12093 raft_consensus.cc:686] T ed6c7c608bf74e968e295585c3c0d1b3 P 5ac1b284ab624af9bcc0235ebb0d2ead [term 1 LEADER]: Becoming Leader. State: Replica: 5ac1b284ab624af9bcc0235ebb0d2ead, State: Running, Role: LEADER
I20260809 09:35:38.929800 12093 consensus_queue.cc:237] T ed6c7c608bf74e968e295585c3c0d1b3 P 5ac1b284ab624af9bcc0235ebb0d2ead [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e326d8d7643f49fba2572694f4a9a3c3" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 46207 } } peers { permanent_uuid: "5ac1b284ab624af9bcc0235ebb0d2ead" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 42417 } } peers { permanent_uuid: "696cb7ca94004c299e1706b18b99498c" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 33021 } }
I20260809 09:35:38.933199 11389 catalog_manager.cc:5495] T ed6c7c608bf74e968e295585c3c0d1b3 P 5ac1b284ab624af9bcc0235ebb0d2ead reported cstate change: term changed from 0 to 1, leader changed from <none> to 5ac1b284ab624af9bcc0235ebb0d2ead (127.20.125.129). New cstate: current_term: 1 leader_uuid: "5ac1b284ab624af9bcc0235ebb0d2ead" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e326d8d7643f49fba2572694f4a9a3c3" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 46207 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5ac1b284ab624af9bcc0235ebb0d2ead" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 42417 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "696cb7ca94004c299e1706b18b99498c" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 33021 } health_report { overall_health: UNKNOWN } } }
I20260809 09:35:38.951960 12080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:35:38.948232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37580 (local address 127.20.125.129:42417)
0809 09:35:38.948381 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0809 09:35:38.948385 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 09:35:38.948400 (+ 15us) server_negotiation.cc:407] Connection header received
0809 09:35:38.948550 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:35:38.948552 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:35:38.948607 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:35:38.948743 (+ 136us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 09:35:38.949002 (+ 259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:35:38.949514 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:35:38.950395 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:35:38.950680 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:35:38.950793 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:35:38.951112 (+ 319us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 09:35:38.951542 (+ 430us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 09:35:38.951680 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0809 09:35:38.951744 (+ 64us) server_negotiation.cc:299] Negotiation successful
0809 09:35:38.951805 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260809 09:35:38.958644 11995 raft_consensus.cc:1261] T ed6c7c608bf74e968e295585c3c0d1b3 P e326d8d7643f49fba2572694f4a9a3c3 [term 1 FOLLOWER]: Refusing update from remote peer 5ac1b284ab624af9bcc0235ebb0d2ead: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 09:35:38.958644 11803 raft_consensus.cc:1261] T ed6c7c608bf74e968e295585c3c0d1b3 P 696cb7ca94004c299e1706b18b99498c [term 1 FOLLOWER]: Refusing update from remote peer 5ac1b284ab624af9bcc0235ebb0d2ead: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 09:35:38.959594 12099 consensus_queue.cc:1035] T ed6c7c608bf74e968e295585c3c0d1b3 P 5ac1b284ab624af9bcc0235ebb0d2ead [LEADER]: Connected to new peer: Peer: permanent_uuid: "e326d8d7643f49fba2572694f4a9a3c3" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 46207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 09:35:38.959822 12093 consensus_queue.cc:1035] T ed6c7c608bf74e968e295585c3c0d1b3 P 5ac1b284ab624af9bcc0235ebb0d2ead [LEADER]: Connected to new peer: Peer: permanent_uuid: "696cb7ca94004c299e1706b18b99498c" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 33021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 09:35:38.968802 12104 mvcc.cc:205] Tried to move back new op lower bound from 7316554297167060992 to 7316554297059885056. Current Snapshot: MvccSnapshot[applied={T|T < 7316554297167060992}]
I20260809 09:35:38.970031 12106 mvcc.cc:205] Tried to move back new op lower bound from 7316554297167060992 to 7316554297059885056. Current Snapshot: MvccSnapshot[applied={T|T < 7316554297167060992}]
I20260809 09:35:38.977241 11389 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56598:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 09:35:38.977407 11389 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56598:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 09:35:38.980070 11389 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 32d6a58804784035826482602ca4a412: Sending DeleteTablet for 3 replicas of tablet ed6c7c608bf74e968e295585c3c0d1b3
I20260809 09:35:38.980954 11969 tablet_service.cc:1514] Processing DeleteTablet for tablet ed6c7c608bf74e968e295585c3c0d1b3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 09:35:38 UTC) from {username='kudu', principal='kudu/127.20.125.190@KRBTEST.COM'} at 127.0.0.1:52746
I20260809 09:35:38.980970 11581 tablet_service.cc:1514] Processing DeleteTablet for tablet ed6c7c608bf74e968e295585c3c0d1b3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 09:35:38 UTC) from {username='kudu', principal='kudu/127.20.125.190@KRBTEST.COM'} at 127.0.0.1:37568
I20260809 09:35:38.981206 11777 tablet_service.cc:1514] Processing DeleteTablet for tablet ed6c7c608bf74e968e295585c3c0d1b3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 09:35:38 UTC) from {username='kudu', principal='kudu/127.20.125.190@KRBTEST.COM'} at 127.0.0.1:34698
I20260809 09:35:38.981424 20982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu with pid 11479
I20260809 09:35:38.995518 12124 tablet_replica.cc:330] T ed6c7c608bf74e968e295585c3c0d1b3 P 696cb7ca94004c299e1706b18b99498c: stopping tablet replica
I20260809 09:35:38.995498 12121 tablet_replica.cc:330] T ed6c7c608bf74e968e295585c3c0d1b3 P e326d8d7643f49fba2572694f4a9a3c3: stopping tablet replica
I20260809 09:35:38.995970 12124 raft_consensus.cc:2229] T ed6c7c608bf74e968e295585c3c0d1b3 P 696cb7ca94004c299e1706b18b99498c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 09:35:38.995999 12121 raft_consensus.cc:2229] T ed6c7c608bf74e968e295585c3c0d1b3 P e326d8d7643f49fba2572694f4a9a3c3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 09:35:38.996382 12121 raft_consensus.cc:2258] T ed6c7c608bf74e968e295585c3c0d1b3 P e326d8d7643f49fba2572694f4a9a3c3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 09:35:38.996371 12124 raft_consensus.cc:2258] T ed6c7c608bf74e968e295585c3c0d1b3 P 696cb7ca94004c299e1706b18b99498c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 09:35:39.001066 12124 ts_tablet_manager.cc:1905] T ed6c7c608bf74e968e295585c3c0d1b3 P 696cb7ca94004c299e1706b18b99498c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 09:35:39.006186 12124 ts_tablet_manager.cc:1918] T ed6c7c608bf74e968e295585c3c0d1b3 P 696cb7ca94004c299e1706b18b99498c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 09:35:39.006340 12124 log.cc:1196] T ed6c7c608bf74e968e295585c3c0d1b3 P 696cb7ca94004c299e1706b18b99498c: Deleting WAL directory at /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/minicluster-data/ts-1/wal/wals/ed6c7c608bf74e968e295585c3c0d1b3
I20260809 09:35:39.006786 12124 ts_tablet_manager.cc:1939] T ed6c7c608bf74e968e295585c3c0d1b3 P 696cb7ca94004c299e1706b18b99498c: Deleting consensus metadata
I20260809 09:35:39.008499 11374 catalog_manager.cc:4841] TS 696cb7ca94004c299e1706b18b99498c (127.20.125.130:33021): tablet ed6c7c608bf74e968e295585c3c0d1b3 (table test-table [id=7dea63d3198940879c82c8c5ceebe710]) successfully deleted
I20260809 09:35:39.009249 12121 ts_tablet_manager.cc:1905] T ed6c7c608bf74e968e295585c3c0d1b3 P e326d8d7643f49fba2572694f4a9a3c3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 09:35:39.011759 12121 ts_tablet_manager.cc:1918] T ed6c7c608bf74e968e295585c3c0d1b3 P e326d8d7643f49fba2572694f4a9a3c3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 09:35:39.011855 12121 log.cc:1196] T ed6c7c608bf74e968e295585c3c0d1b3 P e326d8d7643f49fba2572694f4a9a3c3: Deleting WAL directory at /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/minicluster-data/ts-2/wal/wals/ed6c7c608bf74e968e295585c3c0d1b3
I20260809 09:35:39.012238 12121 ts_tablet_manager.cc:1939] T ed6c7c608bf74e968e295585c3c0d1b3 P e326d8d7643f49fba2572694f4a9a3c3: Deleting consensus metadata
I20260809 09:35:39.013522 11377 catalog_manager.cc:4841] TS e326d8d7643f49fba2572694f4a9a3c3 (127.20.125.131:46207): tablet ed6c7c608bf74e968e295585c3c0d1b3 (table test-table [id=7dea63d3198940879c82c8c5ceebe710]) successfully deleted
W20260809 09:35:39.014390 11378 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.20.125.129:42417 (error 108)
I20260809 09:35:39.014696 20982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu with pid 11683
I20260809 09:35:39.024245 12073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:35:39.023876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.129:42417 (local address 127.0.0.1:37584)
0809 09:35:39.024043 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0809 09:35:39.024147 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.129:42417: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
W20260809 09:35:39.024762 11378 catalog_manager.cc:4584] TS 5ac1b284ab624af9bcc0235ebb0d2ead (127.20.125.129:42417): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ed6c7c608bf74e968e295585c3c0d1b3: Network error: Client connection negotiation failed: client connection to 127.20.125.129:42417: connect: Connection refused (error 111)
I20260809 09:35:39.030344 20982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu with pid 11878
I20260809 09:35:39.044577 20982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu with pid 11352
I20260809 09:35:59.059367 20982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
/tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.125.190:43909
--webserver_interface=127.20.125.190
--webserver_port=41935
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.190.keytab
--principal=kudu/127.20.125.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.125.148:36641
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.125.190:43909
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 09:35:59.177690 12126 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 09:35:59.178020 12126 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 09:35:59.178098 12126 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 09:35:59.182214 12126 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260809 09:35:59.182303 12126 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260809 09:35:59.182332 12126 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260809 09:35:59.182358 12126 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 09:35:59.182416 12126 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 09:35:59.182492 12126 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 09:35:59.182536 12126 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 09:35:59.182571 12126 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 09:35:59.187933 12126 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.125.148:36641
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.20.125.190:43909
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.190.keytab
--principal=kudu/127.20.125.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.125.190:43909
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.125.190
--webserver_port=41935
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12126
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 08:38:55 UTC on ccycloud.quasar-njjleu.root.comops.site
build id 4575935
I20260809 09:35:59.189047 12126 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 09:35:59.190116 12126 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 09:35:59.205468 12126 server_base.cc:1034] running on GCE node
W20260809 09:35:59.205401 12134 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:35:59.205399 12137 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:35:59.205394 12133 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:35:59.206449 12126 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 09:35:59.207769 12126 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 09:35:59.208954 12126 hybrid_clock.cc:648] HybridClock initialized: now 1786268159208919 us; error 50 us; skew 500 ppm
Aug 09 09:35:59 dist-test-slave-7kzw krb5kdc[11335](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268159, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 09:35:59.216955 12126 init.cc:377] Logged in from keytab as kudu/127.20.125.190@KRBTEST.COM (short username kudu)
I20260809 09:35:59.217957 12126 webserver.cc:460] Webserver started at http://127.20.125.190:41935/ using document root <none> and password file <none>
I20260809 09:35:59.218655 12126 fs_manager.cc:359] Metadata directory not provided
I20260809 09:35:59.218779 12126 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 09:35:59.222940 12126 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260809 09:35:59.225274 12147 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 09:35:59.226603 12126 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 09:35:59.226769 12126 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/minicluster-data/master-0/data,/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/minicluster-data/master-0/wal
uuid: "32d6a58804784035826482602ca4a412"
format_stamp: "Formatted at 2026-08-09 09:35:38 on dist-test-slave-7kzw"
server_key: "924dd932b65860656a17b47c1f2507df"
server_key_iv: "ed1d617e6d6196eee8d2a571900da52a"
server_key_version: "encryptionkey@0"
I20260809 09:35:59.227455 12126 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 09:35:59.255662 12126 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 09:35:59.259207 12126 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 09:35:59.259456 12126 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 09:35:59.267419 12126 rpc_server.cc:307] RPC server started. Bound to: 127.20.125.190:43909
I20260809 09:35:59.267750 12226 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.125.190:43909 every 8 connection(s)
I20260809 09:35:59.268682 12126 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/minicluster-data/master-0/data/info.pb
I20260809 09:35:59.268941 20982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu as pid 12126
I20260809 09:35:59.269485 20982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
/tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.125.129:42417
--local_ip_for_outbound_sockets=127.20.125.129
--tserver_master_addrs=127.20.125.190:43909
--webserver_port=35685
--webserver_interface=127.20.125.129
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.129.keytab
--principal=kudu/127.20.125.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.125.148:36641
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 09:35:59.273692 12227 sys_catalog.cc:263] Verifying existing consensus state
I20260809 09:35:59.275569 12227 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 32d6a58804784035826482602ca4a412: Bootstrap starting.
I20260809 09:35:59.293723 12227 log.cc:824] T 00000000000000000000000000000000 P 32d6a58804784035826482602ca4a412: Log is configured to *not* fsync() on all Append() calls
I20260809 09:35:59.303128 12227 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 32d6a58804784035826482602ca4a412: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 09:35:59.303561 12227 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 32d6a58804784035826482602ca4a412: Bootstrap complete.
I20260809 09:35:59.307837 12227 raft_consensus.cc:348] T 00000000000000000000000000000000 P 32d6a58804784035826482602ca4a412 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32d6a58804784035826482602ca4a412" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 43909 } }
I20260809 09:35:59.308295 12227 raft_consensus.cc:729] T 00000000000000000000000000000000 P 32d6a58804784035826482602ca4a412 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32d6a58804784035826482602ca4a412, State: Initialized, Role: FOLLOWER
I20260809 09:35:59.308868 12227 consensus_queue.cc:260] T 00000000000000000000000000000000 P 32d6a58804784035826482602ca4a412 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32d6a58804784035826482602ca4a412" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 43909 } }
I20260809 09:35:59.309028 12227 raft_consensus.cc:388] T 00000000000000000000000000000000 P 32d6a58804784035826482602ca4a412 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 09:35:59.309139 12227 raft_consensus.cc:482] T 00000000000000000000000000000000 P 32d6a58804784035826482602ca4a412 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 09:35:59.309269 12227 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 32d6a58804784035826482602ca4a412 [term 1 FOLLOWER]: Advancing to term 2
I20260809 09:35:59.311374 12227 raft_consensus.cc:504] T 00000000000000000000000000000000 P 32d6a58804784035826482602ca4a412 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32d6a58804784035826482602ca4a412" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 43909 } }
I20260809 09:35:59.311739 12227 leader_election.cc:302] T 00000000000000000000000000000000 P 32d6a58804784035826482602ca4a412 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 32d6a58804784035826482602ca4a412; no voters:
I20260809 09:35:59.312135 12227 leader_election.cc:288] T 00000000000000000000000000000000 P 32d6a58804784035826482602ca4a412 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260809 09:35:59.312235 12231 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 32d6a58804784035826482602ca4a412 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 09:35:59.312484 12231 raft_consensus.cc:686] T 00000000000000000000000000000000 P 32d6a58804784035826482602ca4a412 [term 2 LEADER]: Becoming Leader. State: Replica: 32d6a58804784035826482602ca4a412, State: Running, Role: LEADER
I20260809 09:35:59.312806 12231 consensus_queue.cc:237] T 00000000000000000000000000000000 P 32d6a58804784035826482602ca4a412 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32d6a58804784035826482602ca4a412" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 43909 } }
I20260809 09:35:59.313544 12227 sys_catalog.cc:564] T 00000000000000000000000000000000 P 32d6a58804784035826482602ca4a412 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 09:35:59.314287 12232 sys_catalog.cc:455] T 00000000000000000000000000000000 P 32d6a58804784035826482602ca4a412 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 32d6a58804784035826482602ca4a412. Latest consensus state: current_term: 2 leader_uuid: "32d6a58804784035826482602ca4a412" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32d6a58804784035826482602ca4a412" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 43909 } } }
I20260809 09:35:59.314417 12232 sys_catalog.cc:458] T 00000000000000000000000000000000 P 32d6a58804784035826482602ca4a412 [sys.catalog]: This master's current role is: LEADER
I20260809 09:35:59.314756 12233 sys_catalog.cc:455] T 00000000000000000000000000000000 P 32d6a58804784035826482602ca4a412 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "32d6a58804784035826482602ca4a412" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32d6a58804784035826482602ca4a412" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 43909 } } }
I20260809 09:35:59.314929 12233 sys_catalog.cc:458] T 00000000000000000000000000000000 P 32d6a58804784035826482602ca4a412 [sys.catalog]: This master's current role is: LEADER
I20260809 09:35:59.315255 12237 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 09:35:59.320403 12237 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 09:35:59.321076 12237 catalog_manager.cc:1259] Loaded cluster ID: d57ed51076194ef89d32e3ac81228f04
I20260809 09:35:59.321141 12237 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 09:35:59.323580 12237 catalog_manager.cc:1504] Loading token signing keys...
I20260809 09:35:59.324266 12237 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 32d6a58804784035826482602ca4a412: Loaded TSK: 0
I20260809 09:35:59.340520 12237 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 32d6a58804784035826482602ca4a412: Generated new TSK 1
I20260809 09:35:59.340816 12237 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260809 09:35:59.393447 12229 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 09:35:59.393690 12229 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 09:35:59.393733 12229 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 09:35:59.397768 12229 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 09:35:59.397831 12229 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 09:35:59.397917 12229 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.125.129
I20260809 09:35:59.403095 12229 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.125.148:36641
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.129.keytab
--principal=kudu/127.20.125.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.125.129:42417
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.125.129
--webserver_port=35685
--tserver_master_addrs=127.20.125.190:43909
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12229
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.125.129
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 08:38:55 UTC on ccycloud.quasar-njjleu.root.comops.site
build id 4575935
I20260809 09:35:59.404057 12229 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 09:35:59.405043 12229 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 09:35:59.414443 12260 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:35:59.414443 12262 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:35:59.414530 12264 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:35:59.414983 12229 server_base.cc:1034] running on GCE node
I20260809 09:35:59.415403 12229 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 09:35:59.416142 12229 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 09:35:59.417344 12229 hybrid_clock.cc:648] HybridClock initialized: now 1786268159417330 us; error 36 us; skew 500 ppm
Aug 09 09:35:59 dist-test-slave-7kzw krb5kdc[11335](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268159, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 09:35:59.420261 12229 init.cc:377] Logged in from keytab as kudu/127.20.125.129@KRBTEST.COM (short username kudu)
I20260809 09:35:59.421214 12229 webserver.cc:460] Webserver started at http://127.20.125.129:35685/ using document root <none> and password file <none>
I20260809 09:35:59.421780 12229 fs_manager.cc:359] Metadata directory not provided
I20260809 09:35:59.421857 12229 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 09:35:59.425710 12229 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260809 09:35:59.427979 12275 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 09:35:59.429239 12229 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 09:35:59.429333 12229 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/minicluster-data/ts-0/data,/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/minicluster-data/ts-0/wal
uuid: "5ac1b284ab624af9bcc0235ebb0d2ead"
format_stamp: "Formatted at 2026-08-09 09:35:38 on dist-test-slave-7kzw"
server_key: "60cd82aebd07ab3a9b3bdf5d1ee88114"
server_key_iv: "cc27a2a6d8550174c0980958f956bb28"
server_key_version: "encryptionkey@0"
I20260809 09:35:59.429890 12229 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 09:35:59.460625 12229 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 09:35:59.464430 12229 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 09:35:59.464676 12229 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 09:35:59.465395 12229 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 09:35:59.466612 12288 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260809 09:35:59.471122 12229 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260809 09:35:59.471196 12229 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.000s sys 0.000s
I20260809 09:35:59.471304 12229 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260809 09:35:59.473109 12229 ts_tablet_manager.cc:610] Registered 1 tablets
I20260809 09:35:59.473158 12229 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260809 09:35:59.473481 12288 tablet_bootstrap.cc:492] T ed6c7c608bf74e968e295585c3c0d1b3 P 5ac1b284ab624af9bcc0235ebb0d2ead: Bootstrap starting.
I20260809 09:35:59.484526 12229 rpc_server.cc:307] RPC server started. Bound to: 127.20.125.129:42417
I20260809 09:35:59.484623 12439 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.125.129:42417 every 8 connection(s)
I20260809 09:35:59.485566 12229 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/minicluster-data/ts-0/data/info.pb
I20260809 09:35:59.489279 20982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu as pid 12229
I20260809 09:35:59.489833 20982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
/tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.125.130:33021
--local_ip_for_outbound_sockets=127.20.125.130
--tserver_master_addrs=127.20.125.190:43909
--webserver_port=40129
--webserver_interface=127.20.125.130
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.130.keytab
--principal=kudu/127.20.125.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.125.148:36641
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 09:35:59.493242 12288 log.cc:824] T ed6c7c608bf74e968e295585c3c0d1b3 P 5ac1b284ab624af9bcc0235ebb0d2ead: Log is configured to *not* fsync() on all Append() calls
Aug 09 09:35:59 dist-test-slave-7kzw krb5kdc[11335](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268159, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.129@KRBTEST.COM for kudu/127.20.125.190@KRBTEST.COM
I20260809 09:35:59.501276 12288 tablet_bootstrap.cc:492] T ed6c7c608bf74e968e295585c3c0d1b3 P 5ac1b284ab624af9bcc0235ebb0d2ead: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 09:35:59.501791 12288 tablet_bootstrap.cc:492] T ed6c7c608bf74e968e295585c3c0d1b3 P 5ac1b284ab624af9bcc0235ebb0d2ead: Bootstrap complete.
I20260809 09:35:59.503044 12288 ts_tablet_manager.cc:1397] T ed6c7c608bf74e968e295585c3c0d1b3 P 5ac1b284ab624af9bcc0235ebb0d2ead: Time spent bootstrapping tablet: real 0.030s user 0.014s sys 0.014s
I20260809 09:35:59.506063 12288 raft_consensus.cc:348] T ed6c7c608bf74e968e295585c3c0d1b3 P 5ac1b284ab624af9bcc0235ebb0d2ead [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e326d8d7643f49fba2572694f4a9a3c3" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 46207 } } peers { permanent_uuid: "5ac1b284ab624af9bcc0235ebb0d2ead" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 42417 } } peers { permanent_uuid: "696cb7ca94004c299e1706b18b99498c" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 33021 } }
I20260809 09:35:59.506628 12288 raft_consensus.cc:729] T ed6c7c608bf74e968e295585c3c0d1b3 P 5ac1b284ab624af9bcc0235ebb0d2ead [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ac1b284ab624af9bcc0235ebb0d2ead, State: Initialized, Role: FOLLOWER
I20260809 09:35:59.507190 12288 consensus_queue.cc:260] T ed6c7c608bf74e968e295585c3c0d1b3 P 5ac1b284ab624af9bcc0235ebb0d2ead [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e326d8d7643f49fba2572694f4a9a3c3" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 46207 } } peers { permanent_uuid: "5ac1b284ab624af9bcc0235ebb0d2ead" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 42417 } } peers { permanent_uuid: "696cb7ca94004c299e1706b18b99498c" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 33021 } }
I20260809 09:35:59.507463 12443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:35:59.487814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.129:50831 (local address 127.20.125.190:43909)
0809 09:35:59.488242 (+ 428us) server_negotiation.cc:206] Beginning negotiation
0809 09:35:59.488264 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0809 09:35:59.488696 (+ 432us) server_negotiation.cc:407] Connection header received
0809 09:35:59.490425 (+ 1729us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:35:59.490443 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:35:59.490956 (+ 513us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:35:59.491348 (+ 392us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:35:59.492768 (+ 1420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:35:59.493923 (+ 1155us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:35:59.495944 (+ 2021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:35:59.496647 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:35:59.496832 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:35:59.500425 (+ 3593us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:35:59.500437 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:35:59.500458 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:35:59.500506 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:35:59.503192 (+ 2686us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:35:59.503852 (+ 660us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:35:59.503862 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:35:59.503870 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:35:59.504091 (+ 221us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:35:59.504627 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:35:59.504635 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:35:59.504640 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:35:59.505238 (+ 598us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:35:59.505422 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0809 09:35:59.506059 (+ 637us) server_negotiation.cc:299] Negotiation successful
0809 09:35:59.506386 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":265,"thread_start_us":126,"threads_started":1}
I20260809 09:35:59.507984 12444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:35:59.487769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:43909 (local address 127.20.125.129:50831)
0809 09:35:59.488490 (+ 721us) negotiation.cc:107] Waiting for socket to connect
0809 09:35:59.488552 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0809 09:35:59.489775 (+ 1223us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:35:59.491463 (+ 1688us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:35:59.491476 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:35:59.492047 (+ 571us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:35:59.492547 (+ 500us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:35:59.492562 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:35:59.494101 (+ 1539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:35:59.494109 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:35:59.495781 (+ 1672us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:35:59.495798 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:35:59.496829 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:35:59.496837 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:35:59.497102 (+ 265us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:35:59.497919 (+ 817us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 09:35:59.497952 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:35:59.500174 (+ 2222us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 09:35:59.503332 (+ 3158us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:35:59.503338 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:35:59.503350 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:35:59.503707 (+ 357us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:35:59.504293 (+ 586us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:35:59.504297 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:35:59.504299 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:35:59.504491 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:35:59.505399 (+ 908us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 09:35:59.505406 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 09:35:59.505695 (+ 289us) client_negotiation.cc:769] Sending connection context
0809 09:35:59.505900 (+ 205us) client_negotiation.cc:240] Negotiation successful
0809 09:35:59.506253 (+ 353us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":433,"thread_start_us":152,"threads_started":1}
I20260809 09:35:59.508289 12288 ts_tablet_manager.cc:1428] T ed6c7c608bf74e968e295585c3c0d1b3 P 5ac1b284ab624af9bcc0235ebb0d2ead: Time spent starting tablet: real 0.005s user 0.004s sys 0.001s
I20260809 09:35:59.510531 12440 heartbeater.cc:347] Connected to a master server at 127.20.125.190:43909
I20260809 09:35:59.510773 12440 heartbeater.cc:464] Registering TS with master...
I20260809 09:35:59.511332 12440 heartbeater.cc:511] Master 127.20.125.190:43909 requested a full tablet report, sending...
I20260809 09:35:59.513773 12170 ts_manager.cc:194] Registered new tserver with Master: 5ac1b284ab624af9bcc0235ebb0d2ead (127.20.125.129:42417)
I20260809 09:35:59.517521 12170 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.125.129@KRBTEST.COM'} at 127.20.125.129:50831
I20260809 09:35:59.520423 12440 heartbeater.cc:503] Master 127.20.125.190:43909 was elected leader, sending a full tablet report...
Aug 09 09:35:59 dist-test-slave-7kzw krb5kdc[11335](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268159, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.190@KRBTEST.COM for kudu/127.20.125.129@KRBTEST.COM
I20260809 09:35:59.528016 12452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:35:59.516978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54954 (local address 127.20.125.129:42417)
0809 09:35:59.517870 (+ 892us) server_negotiation.cc:206] Beginning negotiation
0809 09:35:59.517885 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 09:35:59.517909 (+ 24us) server_negotiation.cc:407] Connection header received
0809 09:35:59.517983 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:35:59.517988 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:35:59.518195 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:35:59.518400 (+ 205us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:35:59.519059 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:35:59.519769 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:35:59.520674 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:35:59.521119 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:35:59.521237 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:35:59.524184 (+ 2947us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:35:59.524193 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:35:59.524203 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:35:59.524226 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:35:59.525830 (+ 1604us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:35:59.526423 (+ 593us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:35:59.526428 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:35:59.526429 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:35:59.526574 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:35:59.526939 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:35:59.526942 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:35:59.526944 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:35:59.527192 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:35:59.527312 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0809 09:35:59.527755 (+ 443us) server_negotiation.cc:299] Negotiation successful
0809 09:35:59.527891 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":775,"thread_start_us":176,"threads_started":1}
I20260809 09:35:59.528080 12449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:35:59.516869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.129:42417 (local address 127.0.0.1:54954)
0809 09:35:59.517327 (+ 458us) negotiation.cc:107] Waiting for socket to connect
0809 09:35:59.517385 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0809 09:35:59.517531 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:35:59.518391 (+ 860us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:35:59.518397 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:35:59.518821 (+ 424us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:35:59.518925 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:35:59.518932 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:35:59.519889 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:35:59.519895 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:35:59.520529 (+ 634us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:35:59.520542 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:35:59.521250 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:35:59.521254 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:35:59.521377 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:35:59.522173 (+ 796us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 09:35:59.522190 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:35:59.524038 (+ 1848us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 09:35:59.525950 (+ 1912us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:35:59.525960 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:35:59.525967 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:35:59.526306 (+ 339us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:35:59.526673 (+ 367us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:35:59.526676 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:35:59.526678 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:35:59.526794 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:35:59.527332 (+ 538us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 09:35:59.527339 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 09:35:59.527478 (+ 139us) client_negotiation.cc:769] Sending connection context
0809 09:35:59.527669 (+ 191us) client_negotiation.cc:240] Negotiation successful
0809 09:35:59.527890 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":113,"threads_started":1}
I20260809 09:35:59.529492 12349 tablet_service.cc:1514] Processing DeleteTablet for tablet ed6c7c608bf74e968e295585c3c0d1b3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 09:35:38 UTC) from {username='kudu', principal='kudu/127.20.125.190@KRBTEST.COM'} at 127.0.0.1:54954
I20260809 09:35:59.530414 12454 tablet_replica.cc:330] stopping tablet replica
I20260809 09:35:59.530815 12454 raft_consensus.cc:2229] T ed6c7c608bf74e968e295585c3c0d1b3 P 5ac1b284ab624af9bcc0235ebb0d2ead [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 09:35:59.531101 12454 raft_consensus.cc:2258] T ed6c7c608bf74e968e295585c3c0d1b3 P 5ac1b284ab624af9bcc0235ebb0d2ead [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 09:35:59.532164 12454 ts_tablet_manager.cc:1905] T ed6c7c608bf74e968e295585c3c0d1b3 P 5ac1b284ab624af9bcc0235ebb0d2ead: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 09:35:59.536021 12454 ts_tablet_manager.cc:1918] T ed6c7c608bf74e968e295585c3c0d1b3 P 5ac1b284ab624af9bcc0235ebb0d2ead: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 09:35:59.536108 12454 log.cc:1196] T ed6c7c608bf74e968e295585c3c0d1b3 P 5ac1b284ab624af9bcc0235ebb0d2ead: Deleting WAL directory at /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/minicluster-data/ts-0/wal/wals/ed6c7c608bf74e968e295585c3c0d1b3
I20260809 09:35:59.536381 12454 ts_tablet_manager.cc:1939] T ed6c7c608bf74e968e295585c3c0d1b3 P 5ac1b284ab624af9bcc0235ebb0d2ead: Deleting consensus metadata
I20260809 09:35:59.538229 12151 catalog_manager.cc:4841] TS 5ac1b284ab624af9bcc0235ebb0d2ead (127.20.125.129:42417): tablet ed6c7c608bf74e968e295585c3c0d1b3 (table test-table [id=7dea63d3198940879c82c8c5ceebe710]) successfully deleted
W20260809 09:35:59.619902 12446 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 09:35:59.620186 12446 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 09:35:59.620265 12446 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 09:35:59.624338 12446 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 09:35:59.624423 12446 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 09:35:59.624516 12446 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.125.130
I20260809 09:35:59.629673 12446 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.125.148:36641
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.130.keytab
--principal=kudu/127.20.125.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.125.130:33021
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.125.130
--webserver_port=40129
--tserver_master_addrs=127.20.125.190:43909
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12446
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.125.130
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 08:38:55 UTC on ccycloud.quasar-njjleu.root.comops.site
build id 4575935
I20260809 09:35:59.630790 12446 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 09:35:59.631904 12446 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 09:35:59.642220 12460 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:35:59.642179 12463 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:35:59.642295 12461 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:35:59.642730 12446 server_base.cc:1034] running on GCE node
I20260809 09:35:59.643191 12446 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 09:35:59.643924 12446 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 09:35:59.645138 12446 hybrid_clock.cc:648] HybridClock initialized: now 1786268159645121 us; error 27 us; skew 500 ppm
Aug 09 09:35:59 dist-test-slave-7kzw krb5kdc[11335](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268159, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 09:35:59.648260 12446 init.cc:377] Logged in from keytab as kudu/127.20.125.130@KRBTEST.COM (short username kudu)
I20260809 09:35:59.649277 12446 webserver.cc:460] Webserver started at http://127.20.125.130:40129/ using document root <none> and password file <none>
I20260809 09:35:59.649914 12446 fs_manager.cc:359] Metadata directory not provided
I20260809 09:35:59.650034 12446 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 09:35:59.654260 12446 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260809 09:35:59.656781 12472 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 09:35:59.658088 12446 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 09:35:59.658252 12446 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/minicluster-data/ts-1/data,/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/minicluster-data/ts-1/wal
uuid: "696cb7ca94004c299e1706b18b99498c"
format_stamp: "Formatted at 2026-08-09 09:35:38 on dist-test-slave-7kzw"
server_key: "e4ce44f94462a74aeb53e508794e8739"
server_key_iv: "98bda8f07cfb4c5d370c5a1017a92b61"
server_key_version: "encryptionkey@0"
I20260809 09:35:59.658938 12446 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 09:35:59.681035 12446 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 09:35:59.684903 12446 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 09:35:59.685149 12446 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 09:35:59.685853 12446 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 09:35:59.686877 12446 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 09:35:59.686956 12446 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 09:35:59.687042 12446 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 09:35:59.687101 12446 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 09:35:59.696883 12446 rpc_server.cc:307] RPC server started. Bound to: 127.20.125.130:33021
I20260809 09:35:59.696903 12642 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.125.130:33021 every 8 connection(s)
I20260809 09:35:59.698001 12446 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/minicluster-data/ts-1/data/info.pb
I20260809 09:35:59.699937 20982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu as pid 12446
I20260809 09:35:59.700410 20982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
/tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.125.131:46207
--local_ip_for_outbound_sockets=127.20.125.131
--tserver_master_addrs=127.20.125.190:43909
--webserver_port=38713
--webserver_interface=127.20.125.131
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.131.keytab
--principal=kudu/127.20.125.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.125.148:36641
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 09:35:59 dist-test-slave-7kzw krb5kdc[11335](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268159, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.130@KRBTEST.COM for kudu/127.20.125.190@KRBTEST.COM
I20260809 09:35:59.715721 12443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:35:59.700004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.130:51707 (local address 127.20.125.190:43909)
0809 09:35:59.700817 (+ 813us) server_negotiation.cc:206] Beginning negotiation
0809 09:35:59.700824 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 09:35:59.701885 (+ 1061us) server_negotiation.cc:407] Connection header received
0809 09:35:59.703421 (+ 1536us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:35:59.703424 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:35:59.703479 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:35:59.703570 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:35:59.705138 (+ 1568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:35:59.706113 (+ 975us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:35:59.707607 (+ 1494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:35:59.708067 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:35:59.708209 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:35:59.711393 (+ 3184us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:35:59.711399 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:35:59.711404 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:35:59.711443 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:35:59.713394 (+ 1951us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:35:59.713990 (+ 596us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:35:59.713996 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:35:59.714000 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:35:59.714101 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:35:59.714510 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:35:59.714513 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:35:59.714515 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:35:59.714780 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:35:59.714881 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0809 09:35:59.715382 (+ 501us) server_negotiation.cc:299] Negotiation successful
0809 09:35:59.715530 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":655}
I20260809 09:35:59.716673 12645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:35:59.700469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:43909 (local address 127.20.125.130:51707)
0809 09:35:59.701691 (+ 1222us) negotiation.cc:107] Waiting for socket to connect
0809 09:35:59.701760 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0809 09:35:59.703125 (+ 1365us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:35:59.703809 (+ 684us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:35:59.703822 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:35:59.704412 (+ 590us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:35:59.704949 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:35:59.704965 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:35:59.706301 (+ 1336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:35:59.706304 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:35:59.707462 (+ 1158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:35:59.707474 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:35:59.708259 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:35:59.708262 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:35:59.708421 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:35:59.709197 (+ 776us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 09:35:59.709215 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:35:59.711190 (+ 1975us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 09:35:59.713551 (+ 2361us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:35:59.713559 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:35:59.713582 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:35:59.713860 (+ 278us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:35:59.714201 (+ 341us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:35:59.714204 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:35:59.714211 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:35:59.714353 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:35:59.714886 (+ 533us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 09:35:59.714892 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 09:35:59.715135 (+ 243us) client_negotiation.cc:769] Sending connection context
0809 09:35:59.715292 (+ 157us) client_negotiation.cc:240] Negotiation successful
0809 09:35:59.715659 (+ 367us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":993,"thread_start_us":787,"threads_started":1}
I20260809 09:35:59.717875 12643 heartbeater.cc:347] Connected to a master server at 127.20.125.190:43909
I20260809 09:35:59.718117 12643 heartbeater.cc:464] Registering TS with master...
I20260809 09:35:59.718694 12643 heartbeater.cc:511] Master 127.20.125.190:43909 requested a full tablet report, sending...
I20260809 09:35:59.719828 12170 ts_manager.cc:194] Registered new tserver with Master: 696cb7ca94004c299e1706b18b99498c (127.20.125.130:33021)
I20260809 09:35:59.720534 12170 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.125.130@KRBTEST.COM'} at 127.20.125.130:51707
W20260809 09:35:59.821768 12646 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 09:35:59.822059 12646 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 09:35:59.822147 12646 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 09:35:59.826335 12646 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 09:35:59.826411 12646 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 09:35:59.826537 12646 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.125.131
I20260809 09:35:59.832141 12646 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.125.148:36641
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.131.keytab
--principal=kudu/127.20.125.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.125.131:46207
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.125.131
--webserver_port=38713
--tserver_master_addrs=127.20.125.190:43909
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12646
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.125.131
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 08:38:55 UTC on ccycloud.quasar-njjleu.root.comops.site
build id 4575935
I20260809 09:35:59.833405 12646 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 09:35:59.834753 12646 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 09:35:59.845512 12654 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:35:59.845575 12656 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:35:59.845702 12646 server_base.cc:1034] running on GCE node
W20260809 09:35:59.845561 12660 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:35:59.846155 12646 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 09:35:59.846899 12646 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 09:35:59.848115 12646 hybrid_clock.cc:648] HybridClock initialized: now 1786268159848091 us; error 35 us; skew 500 ppm
Aug 09 09:35:59 dist-test-slave-7kzw krb5kdc[11335](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268159, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 09:35:59.851297 12646 init.cc:377] Logged in from keytab as kudu/127.20.125.131@KRBTEST.COM (short username kudu)
I20260809 09:35:59.852233 12646 webserver.cc:460] Webserver started at http://127.20.125.131:38713/ using document root <none> and password file <none>
I20260809 09:35:59.852780 12646 fs_manager.cc:359] Metadata directory not provided
I20260809 09:35:59.852909 12646 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 09:35:59.857074 12646 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260809 09:35:59.859323 12670 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 09:35:59.860610 12646 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 09:35:59.860715 12646 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/minicluster-data/ts-2/data,/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/minicluster-data/ts-2/wal
uuid: "e326d8d7643f49fba2572694f4a9a3c3"
format_stamp: "Formatted at 2026-08-09 09:35:38 on dist-test-slave-7kzw"
server_key: "b85b39b0443c81ffef29c4c085c8d970"
server_key_iv: "74e02f4580a8d8c897cee5a7de119270"
server_key_version: "encryptionkey@0"
I20260809 09:35:59.861253 12646 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 09:35:59.872838 12646 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 09:35:59.876292 12646 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 09:35:59.876475 12646 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 09:35:59.877092 12646 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 09:35:59.877959 12646 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 09:35:59.878022 12646 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 09:35:59.878069 12646 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 09:35:59.878093 12646 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 09:35:59.888527 12646 rpc_server.cc:307] RPC server started. Bound to: 127.20.125.131:46207
I20260809 09:35:59.888553 12823 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.125.131:46207 every 8 connection(s)
I20260809 09:35:59.889670 12646 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/minicluster-data/ts-2/data/info.pb
Aug 09 09:35:59 dist-test-slave-7kzw krb5kdc[11335](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268159, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.131@KRBTEST.COM for kudu/127.20.125.190@KRBTEST.COM
I20260809 09:35:59.899950 20982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu as pid 12646
I20260809 09:35:59.904984 12443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:35:59.891690 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.131:45435 (local address 127.20.125.190:43909)
0809 09:35:59.891890 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0809 09:35:59.891896 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 09:35:59.892814 (+ 918us) server_negotiation.cc:407] Connection header received
0809 09:35:59.894094 (+ 1280us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:35:59.894097 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:35:59.894150 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:35:59.894252 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:35:59.895491 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:35:59.896070 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:35:59.897331 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:35:59.897556 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:35:59.897664 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:35:59.900575 (+ 2911us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:35:59.900578 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:35:59.900580 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:35:59.900606 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:35:59.902445 (+ 1839us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:35:59.902980 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:35:59.902984 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:35:59.902985 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:35:59.903081 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:35:59.903683 (+ 602us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:35:59.903686 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:35:59.903687 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:35:59.903876 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:35:59.903966 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0809 09:35:59.904657 (+ 691us) server_negotiation.cc:299] Negotiation successful
0809 09:35:59.904814 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260809 09:35:59.906288 12829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:35:59.892127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:43909 (local address 127.20.125.131:45435)
0809 09:35:59.892643 (+ 516us) negotiation.cc:107] Waiting for socket to connect
0809 09:35:59.892687 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 09:35:59.893874 (+ 1187us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:35:59.894416 (+ 542us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:35:59.894422 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:35:59.894852 (+ 430us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:35:59.895333 (+ 481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:35:59.895346 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:35:59.896207 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:35:59.896210 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:35:59.897192 (+ 982us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:35:59.897203 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:35:59.897726 (+ 523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:35:59.897729 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:35:59.897879 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:35:59.898623 (+ 744us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 09:35:59.898640 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:35:59.900379 (+ 1739us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 09:35:59.902585 (+ 2206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:35:59.902591 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:35:59.902605 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:35:59.902878 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:35:59.903255 (+ 377us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:35:59.903260 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:35:59.903262 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:35:59.903501 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:35:59.904000 (+ 499us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 09:35:59.904008 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 09:35:59.904369 (+ 361us) client_negotiation.cc:769] Sending connection context
0809 09:35:59.904583 (+ 214us) client_negotiation.cc:240] Negotiation successful
0809 09:35:59.904957 (+ 374us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":293,"thread_start_us":120,"threads_started":1}
I20260809 09:35:59.907971 12824 heartbeater.cc:347] Connected to a master server at 127.20.125.190:43909
I20260809 09:35:59.908318 12824 heartbeater.cc:464] Registering TS with master...
I20260809 09:35:59.909068 12824 heartbeater.cc:511] Master 127.20.125.190:43909 requested a full tablet report, sending...
I20260809 09:35:59.910413 12170 ts_manager.cc:194] Registered new tserver with Master: e326d8d7643f49fba2572694f4a9a3c3 (127.20.125.131:46207)
I20260809 09:35:59.910935 12832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:35:59.900759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59050 (local address 127.20.125.190:43909)
0809 09:35:59.901071 (+ 312us) server_negotiation.cc:206] Beginning negotiation
0809 09:35:59.901076 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 09:35:59.901189 (+ 113us) server_negotiation.cc:407] Connection header received
0809 09:35:59.901318 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:35:59.901321 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:35:59.901378 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:35:59.901487 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:35:59.902451 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:35:59.903122 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:35:59.904397 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:35:59.904690 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:35:59.904801 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:35:59.906794 (+ 1993us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:35:59.906797 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:35:59.906799 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:35:59.906822 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:35:59.908937 (+ 2115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:35:59.909427 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:35:59.909431 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:35:59.909434 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:35:59.909537 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:35:59.909900 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:35:59.909903 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:35:59.909904 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:35:59.910177 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:35:59.910355 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0809 09:35:59.910607 (+ 252us) server_negotiation.cc:299] Negotiation successful
0809 09:35:59.910731 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":221,"thread_start_us":104,"threads_started":1}
I20260809 09:35:59.911140 12170 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.125.131@KRBTEST.COM'} at 127.20.125.131:45435
I20260809 09:35:59.913028 20982 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 09:35:59.917276 12832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:35:59.913693 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59058 (local address 127.20.125.190:43909)
0809 09:35:59.913832 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0809 09:35:59.913835 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 09:35:59.914045 (+ 210us) server_negotiation.cc:407] Connection header received
0809 09:35:59.914175 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:35:59.914177 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:35:59.914342 (+ 165us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:35:59.914423 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 09:35:59.914716 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:35:59.915274 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:35:59.916214 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:35:59.916437 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:35:59.916526 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:35:59.916843 (+ 317us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 09:35:59.916965 (+ 122us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0809 09:35:59.917176 (+ 211us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:59058: token expired
Metrics: {"server-negotiator.queue_time_us":65}
W20260809 09:35:59.917464 12832 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:59058: token expired
W20260809 09:35:59.917713 12067 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.20.125.190:43909)
I20260809 09:35:59.925429 12832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:35:59.918111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59062 (local address 127.20.125.190:43909)
0809 09:35:59.918259 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0809 09:35:59.918263 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 09:35:59.918277 (+ 14us) server_negotiation.cc:407] Connection header received
0809 09:35:59.918383 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:35:59.918386 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:35:59.918426 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:35:59.918551 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:35:59.919212 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:35:59.919686 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:35:59.920477 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:35:59.920698 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:35:59.920800 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:35:59.922160 (+ 1360us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:35:59.922164 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:35:59.922166 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:35:59.922191 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:35:59.923758 (+ 1567us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:35:59.924203 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:35:59.924206 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:35:59.924208 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:35:59.924296 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:35:59.924576 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:35:59.924580 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:35:59.924581 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:35:59.924767 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:35:59.924904 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0809 09:35:59.925148 (+ 244us) server_negotiation.cc:299] Negotiation successful
0809 09:35:59.925274 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260809 09:35:59.928056 12169 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59062:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 09:35:59.931378 12169 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 09:35:59.939837 12349 tablet_service.cc:1467] Processing CreateTablet for tablet daa0a4a5135b46b7b854d8a99a6198b9 (DEFAULT_TABLE table=test-table [id=2a7b451f27c34e718b780160ef077b88]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 09:35:59.940554 12349 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet daa0a4a5135b46b7b854d8a99a6198b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 09:35:59.943548 12288 tablet_bootstrap.cc:492] T daa0a4a5135b46b7b854d8a99a6198b9 P 5ac1b284ab624af9bcc0235ebb0d2ead: Bootstrap starting.
I20260809 09:35:59.944519 12288 tablet_bootstrap.cc:654] T daa0a4a5135b46b7b854d8a99a6198b9 P 5ac1b284ab624af9bcc0235ebb0d2ead: Neither blocks nor log segments found. Creating new log.
I20260809 09:35:59.944715 12449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:35:59.938644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.130:33021 (local address 127.0.0.1:42204)
0809 09:35:59.938766 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0809 09:35:59.938781 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 09:35:59.938868 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:35:59.939903 (+ 1035us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:35:59.939906 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:35:59.939927 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 09:35:59.940048 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:35:59.940056 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:35:59.941579 (+ 1523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:35:59.941583 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:35:59.942881 (+ 1298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:35:59.942892 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:35:59.944211 (+ 1319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:35:59.944215 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:35:59.944310 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:35:59.944362 (+ 52us) client_negotiation.cc:769] Sending connection context
0809 09:35:59.944456 (+ 94us) client_negotiation.cc:240] Negotiation successful
0809 09:35:59.944566 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46,"mutex_wait_us":47}
I20260809 09:35:59.945076 12840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:35:59.938853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42204 (local address 127.20.125.130:33021)
0809 09:35:59.939372 (+ 519us) server_negotiation.cc:206] Beginning negotiation
0809 09:35:59.939390 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0809 09:35:59.939418 (+ 28us) server_negotiation.cc:407] Connection header received
0809 09:35:59.939507 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:35:59.939517 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:35:59.939738 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:35:59.939929 (+ 191us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 09:35:59.940180 (+ 251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:35:59.941445 (+ 1265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:35:59.943022 (+ 1577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:35:59.944026 (+ 1004us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:35:59.944201 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:35:59.944325 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0809 09:35:59.944506 (+ 181us) server_negotiation.cc:299] Negotiation successful
0809 09:35:59.944709 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":406,"thread_start_us":140,"threads_started":1}
I20260809 09:35:59.945075 12842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:35:59.939228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46978 (local address 127.20.125.131:46207)
0809 09:35:59.939665 (+ 437us) server_negotiation.cc:206] Beginning negotiation
0809 09:35:59.939677 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 09:35:59.939701 (+ 24us) server_negotiation.cc:407] Connection header received
0809 09:35:59.939777 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:35:59.939782 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:35:59.939974 (+ 192us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:35:59.940137 (+ 163us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 09:35:59.940574 (+ 437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:35:59.941819 (+ 1245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:35:59.943074 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:35:59.944021 (+ 947us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:35:59.944172 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:35:59.944325 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0809 09:35:59.944808 (+ 483us) server_negotiation.cc:299] Negotiation successful
0809 09:35:59.944921 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":345,"thread_start_us":153,"threads_started":1}
I20260809 09:35:59.945094 12841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:35:59.938967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.131:46207 (local address 127.0.0.1:46978)
0809 09:35:59.939351 (+ 384us) negotiation.cc:107] Waiting for socket to connect
0809 09:35:59.939366 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 09:35:59.939464 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:35:59.940323 (+ 859us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:35:59.940326 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:35:59.940344 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 09:35:59.940433 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:35:59.940439 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:35:59.941939 (+ 1500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:35:59.941943 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:35:59.942901 (+ 958us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:35:59.942912 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:35:59.944402 (+ 1490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:35:59.944405 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:35:59.944528 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:35:59.944553 (+ 25us) client_negotiation.cc:769] Sending connection context
0809 09:35:59.944629 (+ 76us) client_negotiation.cc:240] Negotiation successful
0809 09:35:59.944703 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":320,"mutex_wait_us":22,"thread_start_us":109,"threads_started":1}
I20260809 09:35:59.946147 12288 tablet_bootstrap.cc:492] T daa0a4a5135b46b7b854d8a99a6198b9 P 5ac1b284ab624af9bcc0235ebb0d2ead: No bootstrap required, opened a new log
I20260809 09:35:59.946249 12288 ts_tablet_manager.cc:1397] T daa0a4a5135b46b7b854d8a99a6198b9 P 5ac1b284ab624af9bcc0235ebb0d2ead: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20260809 09:35:59.946738 12288 raft_consensus.cc:348] T daa0a4a5135b46b7b854d8a99a6198b9 P 5ac1b284ab624af9bcc0235ebb0d2ead [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "696cb7ca94004c299e1706b18b99498c" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 33021 } } peers { permanent_uuid: "e326d8d7643f49fba2572694f4a9a3c3" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 46207 } } peers { permanent_uuid: "5ac1b284ab624af9bcc0235ebb0d2ead" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 42417 } }
I20260809 09:35:59.946877 12288 raft_consensus.cc:374] T daa0a4a5135b46b7b854d8a99a6198b9 P 5ac1b284ab624af9bcc0235ebb0d2ead [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 09:35:59.946918 12288 raft_consensus.cc:729] T daa0a4a5135b46b7b854d8a99a6198b9 P 5ac1b284ab624af9bcc0235ebb0d2ead [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ac1b284ab624af9bcc0235ebb0d2ead, State: Initialized, Role: FOLLOWER
I20260809 09:35:59.947242 12288 consensus_queue.cc:260] T daa0a4a5135b46b7b854d8a99a6198b9 P 5ac1b284ab624af9bcc0235ebb0d2ead [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "696cb7ca94004c299e1706b18b99498c" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 33021 } } peers { permanent_uuid: "e326d8d7643f49fba2572694f4a9a3c3" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 46207 } } peers { permanent_uuid: "5ac1b284ab624af9bcc0235ebb0d2ead" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 42417 } }
I20260809 09:35:59.947865 12288 ts_tablet_manager.cc:1428] T daa0a4a5135b46b7b854d8a99a6198b9 P 5ac1b284ab624af9bcc0235ebb0d2ead: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260809 09:35:59.948572 12729 tablet_service.cc:1467] Processing CreateTablet for tablet daa0a4a5135b46b7b854d8a99a6198b9 (DEFAULT_TABLE table=test-table [id=2a7b451f27c34e718b780160ef077b88]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 09:35:59.948556 12536 tablet_service.cc:1467] Processing CreateTablet for tablet daa0a4a5135b46b7b854d8a99a6198b9 (DEFAULT_TABLE table=test-table [id=2a7b451f27c34e718b780160ef077b88]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 09:35:59.949659 12729 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet daa0a4a5135b46b7b854d8a99a6198b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 09:35:59.949764 12536 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet daa0a4a5135b46b7b854d8a99a6198b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 09:35:59.957206 12845 tablet_bootstrap.cc:492] T daa0a4a5135b46b7b854d8a99a6198b9 P e326d8d7643f49fba2572694f4a9a3c3: Bootstrap starting.
I20260809 09:35:59.957197 12846 tablet_bootstrap.cc:492] T daa0a4a5135b46b7b854d8a99a6198b9 P 696cb7ca94004c299e1706b18b99498c: Bootstrap starting.
I20260809 09:35:59.960111 12845 tablet_bootstrap.cc:654] T daa0a4a5135b46b7b854d8a99a6198b9 P e326d8d7643f49fba2572694f4a9a3c3: Neither blocks nor log segments found. Creating new log.
I20260809 09:35:59.960170 12846 tablet_bootstrap.cc:654] T daa0a4a5135b46b7b854d8a99a6198b9 P 696cb7ca94004c299e1706b18b99498c: Neither blocks nor log segments found. Creating new log.
I20260809 09:35:59.961053 12845 log.cc:824] T daa0a4a5135b46b7b854d8a99a6198b9 P e326d8d7643f49fba2572694f4a9a3c3: Log is configured to *not* fsync() on all Append() calls
I20260809 09:35:59.961140 12846 log.cc:824] T daa0a4a5135b46b7b854d8a99a6198b9 P 696cb7ca94004c299e1706b18b99498c: Log is configured to *not* fsync() on all Append() calls
I20260809 09:35:59.963580 12846 tablet_bootstrap.cc:492] T daa0a4a5135b46b7b854d8a99a6198b9 P 696cb7ca94004c299e1706b18b99498c: No bootstrap required, opened a new log
I20260809 09:35:59.963717 12845 tablet_bootstrap.cc:492] T daa0a4a5135b46b7b854d8a99a6198b9 P e326d8d7643f49fba2572694f4a9a3c3: No bootstrap required, opened a new log
I20260809 09:35:59.963809 12846 ts_tablet_manager.cc:1397] T daa0a4a5135b46b7b854d8a99a6198b9 P 696cb7ca94004c299e1706b18b99498c: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.004s
I20260809 09:35:59.963904 12845 ts_tablet_manager.cc:1397] T daa0a4a5135b46b7b854d8a99a6198b9 P e326d8d7643f49fba2572694f4a9a3c3: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260809 09:35:59.966866 12846 raft_consensus.cc:348] T daa0a4a5135b46b7b854d8a99a6198b9 P 696cb7ca94004c299e1706b18b99498c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "696cb7ca94004c299e1706b18b99498c" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 33021 } } peers { permanent_uuid: "e326d8d7643f49fba2572694f4a9a3c3" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 46207 } } peers { permanent_uuid: "5ac1b284ab624af9bcc0235ebb0d2ead" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 42417 } }
I20260809 09:35:59.967100 12846 raft_consensus.cc:374] T daa0a4a5135b46b7b854d8a99a6198b9 P 696cb7ca94004c299e1706b18b99498c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 09:35:59.967173 12846 raft_consensus.cc:729] T daa0a4a5135b46b7b854d8a99a6198b9 P 696cb7ca94004c299e1706b18b99498c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 696cb7ca94004c299e1706b18b99498c, State: Initialized, Role: FOLLOWER
I20260809 09:35:59.967391 12845 raft_consensus.cc:348] T daa0a4a5135b46b7b854d8a99a6198b9 P e326d8d7643f49fba2572694f4a9a3c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "696cb7ca94004c299e1706b18b99498c" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 33021 } } peers { permanent_uuid: "e326d8d7643f49fba2572694f4a9a3c3" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 46207 } } peers { permanent_uuid: "5ac1b284ab624af9bcc0235ebb0d2ead" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 42417 } }
I20260809 09:35:59.967723 12845 raft_consensus.cc:374] T daa0a4a5135b46b7b854d8a99a6198b9 P e326d8d7643f49fba2572694f4a9a3c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 09:35:59.967793 12845 raft_consensus.cc:729] T daa0a4a5135b46b7b854d8a99a6198b9 P e326d8d7643f49fba2572694f4a9a3c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e326d8d7643f49fba2572694f4a9a3c3, State: Initialized, Role: FOLLOWER
I20260809 09:35:59.967689 12846 consensus_queue.cc:260] T daa0a4a5135b46b7b854d8a99a6198b9 P 696cb7ca94004c299e1706b18b99498c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "696cb7ca94004c299e1706b18b99498c" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 33021 } } peers { permanent_uuid: "e326d8d7643f49fba2572694f4a9a3c3" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 46207 } } peers { permanent_uuid: "5ac1b284ab624af9bcc0235ebb0d2ead" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 42417 } }
I20260809 09:35:59.968254 12845 consensus_queue.cc:260] T daa0a4a5135b46b7b854d8a99a6198b9 P e326d8d7643f49fba2572694f4a9a3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "696cb7ca94004c299e1706b18b99498c" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 33021 } } peers { permanent_uuid: "e326d8d7643f49fba2572694f4a9a3c3" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 46207 } } peers { permanent_uuid: "5ac1b284ab624af9bcc0235ebb0d2ead" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 42417 } }
I20260809 09:35:59.968492 12643 heartbeater.cc:503] Master 127.20.125.190:43909 was elected leader, sending a full tablet report...
I20260809 09:35:59.968852 12846 ts_tablet_manager.cc:1428] T daa0a4a5135b46b7b854d8a99a6198b9 P 696cb7ca94004c299e1706b18b99498c: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260809 09:35:59.969300 12845 ts_tablet_manager.cc:1428] T daa0a4a5135b46b7b854d8a99a6198b9 P e326d8d7643f49fba2572694f4a9a3c3: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260809 09:35:59.969676 12824 heartbeater.cc:503] Master 127.20.125.190:43909 was elected leader, sending a full tablet report...
W20260809 09:35:59.986980 12442 tablet.cc:2369] T daa0a4a5135b46b7b854d8a99a6198b9 P 5ac1b284ab624af9bcc0235ebb0d2ead: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 09:36:00.046600 12850 raft_consensus.cc:482] T daa0a4a5135b46b7b854d8a99a6198b9 P e326d8d7643f49fba2572694f4a9a3c3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 09:36:00.046835 12850 raft_consensus.cc:504] T daa0a4a5135b46b7b854d8a99a6198b9 P e326d8d7643f49fba2572694f4a9a3c3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "696cb7ca94004c299e1706b18b99498c" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 33021 } } peers { permanent_uuid: "e326d8d7643f49fba2572694f4a9a3c3" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 46207 } } peers { permanent_uuid: "5ac1b284ab624af9bcc0235ebb0d2ead" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 42417 } }
I20260809 09:36:00.048046 12850 leader_election.cc:288] T daa0a4a5135b46b7b854d8a99a6198b9 P e326d8d7643f49fba2572694f4a9a3c3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 696cb7ca94004c299e1706b18b99498c (127.20.125.130:33021), 5ac1b284ab624af9bcc0235ebb0d2ead (127.20.125.129:42417)
I20260809 09:36:00.052084 12829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:36:00.048337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.130:33021 (local address 127.20.125.131:37715)
0809 09:36:00.048481 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0809 09:36:00.048497 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 09:36:00.048604 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:36:00.048960 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:36:00.048963 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:36:00.048984 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 09:36:00.049111 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:36:00.049117 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:36:00.049914 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:36:00.049919 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:36:00.050889 (+ 970us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:36:00.050900 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:36:00.051639 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:36:00.051643 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:36:00.051757 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:36:00.051774 (+ 17us) client_negotiation.cc:769] Sending connection context
0809 09:36:00.051865 (+ 91us) client_negotiation.cc:240] Negotiation successful
0809 09:36:00.051939 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50}
I20260809 09:36:00.052084 12840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:36:00.048453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.131:37715 (local address 127.20.125.130:33021)
0809 09:36:00.048575 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0809 09:36:00.048580 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 09:36:00.048606 (+ 26us) server_negotiation.cc:407] Connection header received
0809 09:36:00.048770 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:36:00.048774 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:36:00.048840 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:36:00.048944 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 09:36:00.049231 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:36:00.049780 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:36:00.051057 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:36:00.051497 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:36:00.051639 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:36:00.051696 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0809 09:36:00.051884 (+ 188us) server_negotiation.cc:299] Negotiation successful
0809 09:36:00.051957 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260809 09:36:00.052379 12852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:36:00.048345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.129:42417 (local address 127.20.125.131:45163)
0809 09:36:00.048980 (+ 635us) negotiation.cc:107] Waiting for socket to connect
0809 09:36:00.048991 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 09:36:00.049116 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:36:00.049431 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:36:00.049434 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:36:00.049450 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 09:36:00.049575 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:36:00.049580 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:36:00.050335 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:36:00.050338 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:36:00.051267 (+ 929us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:36:00.051277 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:36:00.051980 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:36:00.051983 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:36:00.052071 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:36:00.052102 (+ 31us) client_negotiation.cc:769] Sending connection context
0809 09:36:00.052164 (+ 62us) client_negotiation.cc:240] Negotiation successful
0809 09:36:00.052232 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":580,"mutex_wait_us":274,"thread_start_us":141,"threads_started":1}
I20260809 09:36:00.052385 12851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:36:00.048426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.131:45163 (local address 127.20.125.129:42417)
0809 09:36:00.048775 (+ 349us) server_negotiation.cc:206] Beginning negotiation
0809 09:36:00.048779 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 09:36:00.049076 (+ 297us) server_negotiation.cc:407] Connection header received
0809 09:36:00.049230 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:36:00.049233 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:36:00.049302 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:36:00.049420 (+ 118us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 09:36:00.049689 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:36:00.050214 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:36:00.051400 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:36:00.051849 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:36:00.051992 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:36:00.052158 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0809 09:36:00.052197 (+ 39us) server_negotiation.cc:299] Negotiation successful
0809 09:36:00.052266 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":270,"thread_start_us":183,"threads_started":1}
I20260809 09:36:00.052891 12571 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "daa0a4a5135b46b7b854d8a99a6198b9" candidate_uuid: "e326d8d7643f49fba2572694f4a9a3c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "696cb7ca94004c299e1706b18b99498c" is_pre_election: true
I20260809 09:36:00.052982 12378 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "daa0a4a5135b46b7b854d8a99a6198b9" candidate_uuid: "e326d8d7643f49fba2572694f4a9a3c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ac1b284ab624af9bcc0235ebb0d2ead" is_pre_election: true
I20260809 09:36:00.053251 12378 raft_consensus.cc:2454] T daa0a4a5135b46b7b854d8a99a6198b9 P 5ac1b284ab624af9bcc0235ebb0d2ead [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e326d8d7643f49fba2572694f4a9a3c3 in term 0.
I20260809 09:36:00.053251 12571 raft_consensus.cc:2454] T daa0a4a5135b46b7b854d8a99a6198b9 P 696cb7ca94004c299e1706b18b99498c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e326d8d7643f49fba2572694f4a9a3c3 in term 0.
I20260809 09:36:00.053731 12674 leader_election.cc:302] T daa0a4a5135b46b7b854d8a99a6198b9 P e326d8d7643f49fba2572694f4a9a3c3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5ac1b284ab624af9bcc0235ebb0d2ead, e326d8d7643f49fba2572694f4a9a3c3; no voters:
I20260809 09:36:00.054008 12850 raft_consensus.cc:2781] T daa0a4a5135b46b7b854d8a99a6198b9 P e326d8d7643f49fba2572694f4a9a3c3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 09:36:00.054127 12850 raft_consensus.cc:482] T daa0a4a5135b46b7b854d8a99a6198b9 P e326d8d7643f49fba2572694f4a9a3c3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 09:36:00.054194 12850 raft_consensus.cc:3037] T daa0a4a5135b46b7b854d8a99a6198b9 P e326d8d7643f49fba2572694f4a9a3c3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 09:36:00.055534 12850 raft_consensus.cc:504] T daa0a4a5135b46b7b854d8a99a6198b9 P e326d8d7643f49fba2572694f4a9a3c3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "696cb7ca94004c299e1706b18b99498c" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 33021 } } peers { permanent_uuid: "e326d8d7643f49fba2572694f4a9a3c3" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 46207 } } peers { permanent_uuid: "5ac1b284ab624af9bcc0235ebb0d2ead" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 42417 } }
I20260809 09:36:00.056010 12850 leader_election.cc:288] T daa0a4a5135b46b7b854d8a99a6198b9 P e326d8d7643f49fba2572694f4a9a3c3 [CANDIDATE]: Term 1 election: Requested vote from peers 696cb7ca94004c299e1706b18b99498c (127.20.125.130:33021), 5ac1b284ab624af9bcc0235ebb0d2ead (127.20.125.129:42417)
I20260809 09:36:00.056373 12571 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "daa0a4a5135b46b7b854d8a99a6198b9" candidate_uuid: "e326d8d7643f49fba2572694f4a9a3c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "696cb7ca94004c299e1706b18b99498c"
I20260809 09:36:00.056406 12378 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "daa0a4a5135b46b7b854d8a99a6198b9" candidate_uuid: "e326d8d7643f49fba2572694f4a9a3c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ac1b284ab624af9bcc0235ebb0d2ead"
I20260809 09:36:00.056519 12378 raft_consensus.cc:3037] T daa0a4a5135b46b7b854d8a99a6198b9 P 5ac1b284ab624af9bcc0235ebb0d2ead [term 0 FOLLOWER]: Advancing to term 1
I20260809 09:36:00.056514 12571 raft_consensus.cc:3037] T daa0a4a5135b46b7b854d8a99a6198b9 P 696cb7ca94004c299e1706b18b99498c [term 0 FOLLOWER]: Advancing to term 1
I20260809 09:36:00.058068 12378 raft_consensus.cc:2454] T daa0a4a5135b46b7b854d8a99a6198b9 P 5ac1b284ab624af9bcc0235ebb0d2ead [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e326d8d7643f49fba2572694f4a9a3c3 in term 1.
I20260809 09:36:00.058068 12571 raft_consensus.cc:2454] T daa0a4a5135b46b7b854d8a99a6198b9 P 696cb7ca94004c299e1706b18b99498c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e326d8d7643f49fba2572694f4a9a3c3 in term 1.
I20260809 09:36:00.058432 12671 leader_election.cc:302] T daa0a4a5135b46b7b854d8a99a6198b9 P e326d8d7643f49fba2572694f4a9a3c3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 696cb7ca94004c299e1706b18b99498c, e326d8d7643f49fba2572694f4a9a3c3; no voters:
I20260809 09:36:00.058692 12850 raft_consensus.cc:2781] T daa0a4a5135b46b7b854d8a99a6198b9 P e326d8d7643f49fba2572694f4a9a3c3 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 09:36:00.058948 12850 raft_consensus.cc:686] T daa0a4a5135b46b7b854d8a99a6198b9 P e326d8d7643f49fba2572694f4a9a3c3 [term 1 LEADER]: Becoming Leader. State: Replica: e326d8d7643f49fba2572694f4a9a3c3, State: Running, Role: LEADER
I20260809 09:36:00.059352 12850 consensus_queue.cc:237] T daa0a4a5135b46b7b854d8a99a6198b9 P e326d8d7643f49fba2572694f4a9a3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "696cb7ca94004c299e1706b18b99498c" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 33021 } } peers { permanent_uuid: "e326d8d7643f49fba2572694f4a9a3c3" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 46207 } } peers { permanent_uuid: "5ac1b284ab624af9bcc0235ebb0d2ead" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 42417 } }
I20260809 09:36:00.062965 12170 catalog_manager.cc:5495] T daa0a4a5135b46b7b854d8a99a6198b9 P e326d8d7643f49fba2572694f4a9a3c3 reported cstate change: term changed from 0 to 1, leader changed from <none> to e326d8d7643f49fba2572694f4a9a3c3 (127.20.125.131). New cstate: current_term: 1 leader_uuid: "e326d8d7643f49fba2572694f4a9a3c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "696cb7ca94004c299e1706b18b99498c" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 33021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e326d8d7643f49fba2572694f4a9a3c3" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 46207 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5ac1b284ab624af9bcc0235ebb0d2ead" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 42417 } health_report { overall_health: UNKNOWN } } }
I20260809 09:36:00.102236 12842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:36:00.098400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46988 (local address 127.20.125.131:46207)
0809 09:36:00.098573 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0809 09:36:00.098577 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 09:36:00.098592 (+ 15us) server_negotiation.cc:407] Connection header received
0809 09:36:00.098724 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:36:00.098727 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:36:00.098788 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:36:00.098883 (+ 95us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 09:36:00.099106 (+ 223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:36:00.099657 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:36:00.100506 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:36:00.100750 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:36:00.100844 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:36:00.101200 (+ 356us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 09:36:00.101639 (+ 439us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 09:36:00.101739 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0809 09:36:00.101943 (+ 204us) server_negotiation.cc:299] Negotiation successful
0809 09:36:00.102036 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260809 09:36:00.108911 12378 raft_consensus.cc:1261] T daa0a4a5135b46b7b854d8a99a6198b9 P 5ac1b284ab624af9bcc0235ebb0d2ead [term 1 FOLLOWER]: Refusing update from remote peer e326d8d7643f49fba2572694f4a9a3c3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 09:36:00.108914 12571 raft_consensus.cc:1261] T daa0a4a5135b46b7b854d8a99a6198b9 P 696cb7ca94004c299e1706b18b99498c [term 1 FOLLOWER]: Refusing update from remote peer e326d8d7643f49fba2572694f4a9a3c3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 09:36:00.109742 12850 consensus_queue.cc:1035] T daa0a4a5135b46b7b854d8a99a6198b9 P e326d8d7643f49fba2572694f4a9a3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "696cb7ca94004c299e1706b18b99498c" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 33021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 09:36:00.110013 12856 consensus_queue.cc:1035] T daa0a4a5135b46b7b854d8a99a6198b9 P e326d8d7643f49fba2572694f4a9a3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5ac1b284ab624af9bcc0235ebb0d2ead" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 42417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 09:36:00.117282 12863 mvcc.cc:205] Tried to move back new op lower bound from 7316554383797673984 to 7316554383606222848. Current Snapshot: MvccSnapshot[applied={T|T < 7316554383797673984}]
I20260809 09:36:00.127411 12170 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59062:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 09:36:00.127624 12170 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59062:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 09:36:00.130419 12170 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 32d6a58804784035826482602ca4a412: Sending DeleteTablet for 3 replicas of tablet daa0a4a5135b46b7b854d8a99a6198b9
I20260809 09:36:00.131302 12536 tablet_service.cc:1514] Processing DeleteTablet for tablet daa0a4a5135b46b7b854d8a99a6198b9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 09:36:00 UTC) from {username='kudu', principal='kudu/127.20.125.190@KRBTEST.COM'} at 127.0.0.1:42204
I20260809 09:36:00.131364 12729 tablet_service.cc:1514] Processing DeleteTablet for tablet daa0a4a5135b46b7b854d8a99a6198b9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 09:36:00 UTC) from {username='kudu', principal='kudu/127.20.125.190@KRBTEST.COM'} at 127.0.0.1:46978
I20260809 09:36:00.131467 12349 tablet_service.cc:1514] Processing DeleteTablet for tablet daa0a4a5135b46b7b854d8a99a6198b9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 09:36:00 UTC) from {username='kudu', principal='kudu/127.20.125.190@KRBTEST.COM'} at 127.0.0.1:54954
I20260809 09:36:00.131937 12874 tablet_replica.cc:330] T daa0a4a5135b46b7b854d8a99a6198b9 P e326d8d7643f49fba2572694f4a9a3c3: stopping tablet replica
I20260809 09:36:00.132237 12872 tablet_replica.cc:330] T daa0a4a5135b46b7b854d8a99a6198b9 P 696cb7ca94004c299e1706b18b99498c: stopping tablet replica
I20260809 09:36:00.132308 20982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu with pid 12229
I20260809 09:36:00.132401 12874 raft_consensus.cc:2229] T daa0a4a5135b46b7b854d8a99a6198b9 P e326d8d7643f49fba2572694f4a9a3c3 [term 1 LEADER]: Raft consensus shutting down.
I20260809 09:36:00.132542 12872 raft_consensus.cc:2229] T daa0a4a5135b46b7b854d8a99a6198b9 P 696cb7ca94004c299e1706b18b99498c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 09:36:00.132872 12874 raft_consensus.cc:2258] T daa0a4a5135b46b7b854d8a99a6198b9 P e326d8d7643f49fba2572694f4a9a3c3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 09:36:00.132893 12872 raft_consensus.cc:2258] T daa0a4a5135b46b7b854d8a99a6198b9 P 696cb7ca94004c299e1706b18b99498c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 09:36:00.136368 12872 ts_tablet_manager.cc:1905] T daa0a4a5135b46b7b854d8a99a6198b9 P 696cb7ca94004c299e1706b18b99498c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 09:36:00.139434 12872 ts_tablet_manager.cc:1918] T daa0a4a5135b46b7b854d8a99a6198b9 P 696cb7ca94004c299e1706b18b99498c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 09:36:00.139539 12872 log.cc:1196] T daa0a4a5135b46b7b854d8a99a6198b9 P 696cb7ca94004c299e1706b18b99498c: Deleting WAL directory at /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/minicluster-data/ts-1/wal/wals/daa0a4a5135b46b7b854d8a99a6198b9
I20260809 09:36:00.139904 12872 ts_tablet_manager.cc:1939] T daa0a4a5135b46b7b854d8a99a6198b9 P 696cb7ca94004c299e1706b18b99498c: Deleting consensus metadata
I20260809 09:36:00.141379 12148 catalog_manager.cc:4841] TS 696cb7ca94004c299e1706b18b99498c (127.20.125.130:33021): tablet daa0a4a5135b46b7b854d8a99a6198b9 (table test-table [id=2a7b451f27c34e718b780160ef077b88]) successfully deleted
I20260809 09:36:00.142421 12874 ts_tablet_manager.cc:1905] T daa0a4a5135b46b7b854d8a99a6198b9 P e326d8d7643f49fba2572694f4a9a3c3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 09:36:00.145459 12874 ts_tablet_manager.cc:1918] T daa0a4a5135b46b7b854d8a99a6198b9 P e326d8d7643f49fba2572694f4a9a3c3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 09:36:00.145573 12874 log.cc:1196] T daa0a4a5135b46b7b854d8a99a6198b9 P e326d8d7643f49fba2572694f4a9a3c3: Deleting WAL directory at /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/minicluster-data/ts-2/wal/wals/daa0a4a5135b46b7b854d8a99a6198b9
I20260809 09:36:00.145942 12874 ts_tablet_manager.cc:1939] T daa0a4a5135b46b7b854d8a99a6198b9 P e326d8d7643f49fba2572694f4a9a3c3: Deleting consensus metadata
I20260809 09:36:00.147426 12150 catalog_manager.cc:4841] TS e326d8d7643f49fba2572694f4a9a3c3 (127.20.125.131:46207): tablet daa0a4a5135b46b7b854d8a99a6198b9 (table test-table [id=2a7b451f27c34e718b780160ef077b88]) successfully deleted
W20260809 09:36:00.156718 12151 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.20.125.129:42417 (error 108)
I20260809 09:36:00.157375 20982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu with pid 12446
I20260809 09:36:00.160074 12841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:36:00.159631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.129:42417 (local address 127.0.0.1:54964)
0809 09:36:00.159869 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0809 09:36:00.159955 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.129:42417: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133}
W20260809 09:36:00.160426 12151 catalog_manager.cc:4584] TS 5ac1b284ab624af9bcc0235ebb0d2ead (127.20.125.129:42417): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet daa0a4a5135b46b7b854d8a99a6198b9: Network error: Client connection negotiation failed: client connection to 127.20.125.129:42417: connect: Connection refused (error 111)
I20260809 09:36:00.168015 20982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu with pid 12646
I20260809 09:36:00.181902 20982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu with pid 12126
I20260809 09:36:20.198151 20982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
/tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.125.190:43909
--webserver_interface=127.20.125.190
--webserver_port=41935
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.190.keytab
--principal=kudu/127.20.125.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.125.148:36641
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.125.190:43909
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 09:36:20.313004 12876 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 09:36:20.313271 12876 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 09:36:20.313313 12876 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 09:36:20.317297 12876 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260809 09:36:20.317349 12876 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260809 09:36:20.317364 12876 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260809 09:36:20.317378 12876 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 09:36:20.317395 12876 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 09:36:20.317443 12876 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 09:36:20.317478 12876 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 09:36:20.317498 12876 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 09:36:20.322805 12876 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.125.148:36641
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.20.125.190:43909
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.190.keytab
--principal=kudu/127.20.125.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.125.190:43909
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.125.190
--webserver_port=41935
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12876
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 08:38:55 UTC on ccycloud.quasar-njjleu.root.comops.site
build id 4575935
I20260809 09:36:20.323845 12876 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 09:36:20.324882 12876 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 09:36:20.333918 12886 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:36:20.334012 12876 server_base.cc:1034] running on GCE node
W20260809 09:36:20.333964 12883 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:36:20.333940 12884 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:36:20.334909 12876 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 09:36:20.336011 12876 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 09:36:20.337229 12876 hybrid_clock.cc:648] HybridClock initialized: now 1786268180337215 us; error 26 us; skew 500 ppm
Aug 09 09:36:20 dist-test-slave-7kzw krb5kdc[11335](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268180, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 09:36:20.340093 12876 init.cc:377] Logged in from keytab as kudu/127.20.125.190@KRBTEST.COM (short username kudu)
I20260809 09:36:20.340962 12876 webserver.cc:460] Webserver started at http://127.20.125.190:41935/ using document root <none> and password file <none>
I20260809 09:36:20.341516 12876 fs_manager.cc:359] Metadata directory not provided
I20260809 09:36:20.341612 12876 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 09:36:20.345440 12876 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260809 09:36:20.347565 12895 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 09:36:20.348796 12876 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 09:36:20.348938 12876 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/minicluster-data/master-0/data,/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/minicluster-data/master-0/wal
uuid: "32d6a58804784035826482602ca4a412"
format_stamp: "Formatted at 2026-08-09 09:35:38 on dist-test-slave-7kzw"
server_key: "924dd932b65860656a17b47c1f2507df"
server_key_iv: "ed1d617e6d6196eee8d2a571900da52a"
server_key_version: "encryptionkey@0"
I20260809 09:36:20.349462 12876 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 09:36:20.368692 12876 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 09:36:20.371822 12876 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 09:36:20.372045 12876 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 09:36:20.379092 12876 rpc_server.cc:307] RPC server started. Bound to: 127.20.125.190:43909
I20260809 09:36:20.379383 12968 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.125.190:43909 every 8 connection(s)
I20260809 09:36:20.380193 12876 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/minicluster-data/master-0/data/info.pb
I20260809 09:36:20.384424 12969 sys_catalog.cc:263] Verifying existing consensus state
I20260809 09:36:20.386059 12969 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 32d6a58804784035826482602ca4a412: Bootstrap starting.
I20260809 09:36:20.386092 20982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu as pid 12876
I20260809 09:36:20.386719 20982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
/tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.125.129:42417
--local_ip_for_outbound_sockets=127.20.125.129
--tserver_master_addrs=127.20.125.190:43909
--webserver_port=35685
--webserver_interface=127.20.125.129
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.129.keytab
--principal=kudu/127.20.125.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.125.148:36641
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 09:36:20.410060 12969 log.cc:824] T 00000000000000000000000000000000 P 32d6a58804784035826482602ca4a412: Log is configured to *not* fsync() on all Append() calls
I20260809 09:36:20.424723 12969 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 32d6a58804784035826482602ca4a412: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 09:36:20.425218 12969 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 32d6a58804784035826482602ca4a412: Bootstrap complete.
I20260809 09:36:20.430819 12969 raft_consensus.cc:348] T 00000000000000000000000000000000 P 32d6a58804784035826482602ca4a412 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32d6a58804784035826482602ca4a412" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 43909 } }
I20260809 09:36:20.431387 12969 raft_consensus.cc:729] T 00000000000000000000000000000000 P 32d6a58804784035826482602ca4a412 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32d6a58804784035826482602ca4a412, State: Initialized, Role: FOLLOWER
I20260809 09:36:20.432003 12969 consensus_queue.cc:260] T 00000000000000000000000000000000 P 32d6a58804784035826482602ca4a412 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32d6a58804784035826482602ca4a412" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 43909 } }
I20260809 09:36:20.432155 12969 raft_consensus.cc:388] T 00000000000000000000000000000000 P 32d6a58804784035826482602ca4a412 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 09:36:20.432256 12969 raft_consensus.cc:482] T 00000000000000000000000000000000 P 32d6a58804784035826482602ca4a412 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 09:36:20.432363 12969 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 32d6a58804784035826482602ca4a412 [term 2 FOLLOWER]: Advancing to term 3
I20260809 09:36:20.434926 12969 raft_consensus.cc:504] T 00000000000000000000000000000000 P 32d6a58804784035826482602ca4a412 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32d6a58804784035826482602ca4a412" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 43909 } }
I20260809 09:36:20.435307 12969 leader_election.cc:302] T 00000000000000000000000000000000 P 32d6a58804784035826482602ca4a412 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 32d6a58804784035826482602ca4a412; no voters:
I20260809 09:36:20.435738 12969 leader_election.cc:288] T 00000000000000000000000000000000 P 32d6a58804784035826482602ca4a412 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260809 09:36:20.435868 12975 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 32d6a58804784035826482602ca4a412 [term 3 FOLLOWER]: Leader election won for term 3
I20260809 09:36:20.436174 12975 raft_consensus.cc:686] T 00000000000000000000000000000000 P 32d6a58804784035826482602ca4a412 [term 3 LEADER]: Becoming Leader. State: Replica: 32d6a58804784035826482602ca4a412, State: Running, Role: LEADER
I20260809 09:36:20.436499 12975 consensus_queue.cc:237] T 00000000000000000000000000000000 P 32d6a58804784035826482602ca4a412 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32d6a58804784035826482602ca4a412" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 43909 } }
I20260809 09:36:20.437028 12969 sys_catalog.cc:564] T 00000000000000000000000000000000 P 32d6a58804784035826482602ca4a412 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 09:36:20.438091 12977 sys_catalog.cc:455] T 00000000000000000000000000000000 P 32d6a58804784035826482602ca4a412 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 32d6a58804784035826482602ca4a412. Latest consensus state: current_term: 3 leader_uuid: "32d6a58804784035826482602ca4a412" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32d6a58804784035826482602ca4a412" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 43909 } } }
I20260809 09:36:20.438263 12977 sys_catalog.cc:458] T 00000000000000000000000000000000 P 32d6a58804784035826482602ca4a412 [sys.catalog]: This master's current role is: LEADER
I20260809 09:36:20.438092 12976 sys_catalog.cc:455] T 00000000000000000000000000000000 P 32d6a58804784035826482602ca4a412 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "32d6a58804784035826482602ca4a412" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32d6a58804784035826482602ca4a412" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 43909 } } }
I20260809 09:36:20.438364 12976 sys_catalog.cc:458] T 00000000000000000000000000000000 P 32d6a58804784035826482602ca4a412 [sys.catalog]: This master's current role is: LEADER
I20260809 09:36:20.439025 12986 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 09:36:20.444341 12986 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 09:36:20.444993 12986 catalog_manager.cc:1259] Loaded cluster ID: d57ed51076194ef89d32e3ac81228f04
I20260809 09:36:20.445057 12986 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 09:36:20.447573 12986 catalog_manager.cc:1504] Loading token signing keys...
I20260809 09:36:20.448328 12986 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 32d6a58804784035826482602ca4a412: Loaded TSK: 0
I20260809 09:36:20.448417 12986 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 32d6a58804784035826482602ca4a412: Loaded TSK: 1
F20260809 09:36:20.449301 12986 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f6d22c469ad google::LogMessage::Fail() at ??:0
@ 0x7f6d22c4988a google::LogMessage::SendToLog() at ??:0
@ 0x7f6d22c464de google::LogMessage::Flush() at ??:0
@ 0x7f6d22c47a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f6d2590a18d kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f6d25902f0d kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f6d2d5af7bb kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f6d2d58999c kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f6d2d58a178 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlvE3_clEv at ??:0
@ 0x7f6d2d5cbdf8 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7f6d2d5ea82b std::function<>::operator()() at ??:0
@ 0x7f6d2d589b4f _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_ at ??:0
@ 0x7f6d2d58af3e kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f6d2d587579 _ZZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvENKUlvE_clEv at ??:0
@ 0x7f6d2d5cb640 _ZNSt17_Function_handlerIFvvEZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f6d24e983d7 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f6d24e98c83 _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7f6d24e9a3f2 _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f6d24e89512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f6d264cd6db start_thread at ??:0
@ 0x7f6d2174461f clone at ??:0
W20260809 09:36:20.560388 12971 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 09:36:20.560655 12971 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 09:36:20.560700 12971 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 09:36:20.564859 12971 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 09:36:20.564921 12971 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 09:36:20.565016 12971 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.125.129
I20260809 09:36:20.570457 12971 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.125.148:36641
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.129.keytab
--principal=kudu/127.20.125.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.125.129:42417
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.125.129
--webserver_port=35685
--tserver_master_addrs=127.20.125.190:43909
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12971
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.125.129
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 08:38:55 UTC on ccycloud.quasar-njjleu.root.comops.site
build id 4575935
I20260809 09:36:20.571678 12971 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 09:36:20.572752 12971 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 09:36:20.583804 13037 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:36:20.583915 13039 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:36:20.583946 13036 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:36:20.584291 12971 server_base.cc:1034] running on GCE node
I20260809 09:36:20.584719 12971 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 09:36:20.585500 12971 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 09:36:20.586683 12971 hybrid_clock.cc:648] HybridClock initialized: now 1786268180586648 us; error 50 us; skew 500 ppm
Aug 09 09:36:20 dist-test-slave-7kzw krb5kdc[11335](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268180, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 09:36:20.589884 12971 init.cc:377] Logged in from keytab as kudu/127.20.125.129@KRBTEST.COM (short username kudu)
I20260809 09:36:20.590826 12971 webserver.cc:460] Webserver started at http://127.20.125.129:35685/ using document root <none> and password file <none>
I20260809 09:36:20.591401 12971 fs_manager.cc:359] Metadata directory not provided
I20260809 09:36:20.591495 12971 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 09:36:20.595501 12971 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260809 09:36:20.600071 13048 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 09:36:20.601466 12971 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 09:36:20.601689 12971 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/minicluster-data/ts-0/data,/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/minicluster-data/ts-0/wal
uuid: "5ac1b284ab624af9bcc0235ebb0d2ead"
format_stamp: "Formatted at 2026-08-09 09:35:38 on dist-test-slave-7kzw"
server_key: "60cd82aebd07ab3a9b3bdf5d1ee88114"
server_key_iv: "cc27a2a6d8550174c0980958f956bb28"
server_key_version: "encryptionkey@0"
I20260809 09:36:20.602262 12971 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 09:36:20.618283 12971 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 09:36:20.622143 12971 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 09:36:20.622395 12971 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 09:36:20.623153 12971 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 09:36:20.624416 13058 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260809 09:36:20.630255 12971 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260809 09:36:20.630368 12971 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.001s sys 0.000s
I20260809 09:36:20.630511 12971 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260809 09:36:20.632449 12971 ts_tablet_manager.cc:610] Registered 1 tablets
I20260809 09:36:20.632530 12971 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260809 09:36:20.632835 13058 tablet_bootstrap.cc:492] T daa0a4a5135b46b7b854d8a99a6198b9 P 5ac1b284ab624af9bcc0235ebb0d2ead: Bootstrap starting.
I20260809 09:36:20.649077 12971 rpc_server.cc:307] RPC server started. Bound to: 127.20.125.129:42417
I20260809 09:36:20.650605 12971 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/minicluster-data/ts-0/data/info.pb
I20260809 09:36:20.652308 13205 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.125.129:42417 every 8 connection(s)
I20260809 09:36:20.658576 20982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu as pid 12971
I20260809 09:36:20.659179 20982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
/tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.125.130:33021
--local_ip_for_outbound_sockets=127.20.125.130
--tserver_master_addrs=127.20.125.190:43909
--webserver_port=40129
--webserver_interface=127.20.125.130
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.130.keytab
--principal=kudu/127.20.125.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.125.148:36641
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 09:36:20.660061 13058 log.cc:824] T daa0a4a5135b46b7b854d8a99a6198b9 P 5ac1b284ab624af9bcc0235ebb0d2ead: Log is configured to *not* fsync() on all Append() calls
I20260809 09:36:20.669163 13058 tablet_bootstrap.cc:492] T daa0a4a5135b46b7b854d8a99a6198b9 P 5ac1b284ab624af9bcc0235ebb0d2ead: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 09:36:20.669646 13058 tablet_bootstrap.cc:492] T daa0a4a5135b46b7b854d8a99a6198b9 P 5ac1b284ab624af9bcc0235ebb0d2ead: Bootstrap complete.
I20260809 09:36:20.671185 13058 ts_tablet_manager.cc:1397] T daa0a4a5135b46b7b854d8a99a6198b9 P 5ac1b284ab624af9bcc0235ebb0d2ead: Time spent bootstrapping tablet: real 0.039s user 0.014s sys 0.020s
I20260809 09:36:20.674587 13058 raft_consensus.cc:348] T daa0a4a5135b46b7b854d8a99a6198b9 P 5ac1b284ab624af9bcc0235ebb0d2ead [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "696cb7ca94004c299e1706b18b99498c" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 33021 } } peers { permanent_uuid: "e326d8d7643f49fba2572694f4a9a3c3" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 46207 } } peers { permanent_uuid: "5ac1b284ab624af9bcc0235ebb0d2ead" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 42417 } }
I20260809 09:36:20.675137 13058 raft_consensus.cc:729] T daa0a4a5135b46b7b854d8a99a6198b9 P 5ac1b284ab624af9bcc0235ebb0d2ead [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ac1b284ab624af9bcc0235ebb0d2ead, State: Initialized, Role: FOLLOWER
I20260809 09:36:20.675701 13058 consensus_queue.cc:260] T daa0a4a5135b46b7b854d8a99a6198b9 P 5ac1b284ab624af9bcc0235ebb0d2ead [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "696cb7ca94004c299e1706b18b99498c" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 33021 } } peers { permanent_uuid: "e326d8d7643f49fba2572694f4a9a3c3" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 46207 } } peers { permanent_uuid: "5ac1b284ab624af9bcc0235ebb0d2ead" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 42417 } }
I20260809 09:36:20.676826 13058 ts_tablet_manager.cc:1428] T daa0a4a5135b46b7b854d8a99a6198b9 P 5ac1b284ab624af9bcc0235ebb0d2ead: Time spent starting tablet: real 0.006s user 0.005s sys 0.001s
W20260809 09:36:20.698212 13209 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 09:36:20.653841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:43909 (local address 127.20.125.129:34511)
0809 09:36:20.655227 (+ 1386us) negotiation.cc:107] Waiting for socket to connect
0809 09:36:20.655299 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0809 09:36:20.656530 (+ 1231us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:36:20.697191 (+ 40661us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.190:43909: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":1115,"thread_start_us":148,"threads_started":1}
W20260809 09:36:20.698800 13206 heartbeater.cc:644] Failed to heartbeat to 127.20.125.190:43909 (0 consecutive failures): Network error: Failed to ping master at 127.20.125.190:43909: Client connection negotiation failed: client connection to 127.20.125.190:43909: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260809 09:36:20.699589 13209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:36:20.699292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:43909 (local address 127.20.125.129:46145)
0809 09:36:20.699432 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0809 09:36:20.699516 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.190:43909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260809 09:36:20.700356 13209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:36:20.700113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:43909 (local address 127.20.125.129:55039)
0809 09:36:20.700240 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0809 09:36:20.700286 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.190:43909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260809 09:36:20.700579 13206 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 09:36:20.804889 13213 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 09:36:20.805219 13213 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 09:36:20.805301 13213 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 09:36:20.809523 13213 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 09:36:20.809651 13213 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 09:36:20.809715 13213 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.125.130
I20260809 09:36:20.815143 13213 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.125.148:36641
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.130.keytab
--principal=kudu/127.20.125.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.125.130:33021
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.125.130
--webserver_port=40129
--tserver_master_addrs=127.20.125.190:43909
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13213
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.125.130
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 08:38:55 UTC on ccycloud.quasar-njjleu.root.comops.site
build id 4575935
I20260809 09:36:20.816197 13213 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 09:36:20.817196 13213 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 09:36:20.827631 13213 server_base.cc:1034] running on GCE node
W20260809 09:36:20.827589 13222 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:36:20.827605 13221 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:36:20.827822 13225 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:36:20.828397 13213 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 09:36:20.829066 13213 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 09:36:20.830248 13213 hybrid_clock.cc:648] HybridClock initialized: now 1786268180830231 us; error 32 us; skew 500 ppm
Aug 09 09:36:20 dist-test-slave-7kzw krb5kdc[11335](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268180, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 09:36:20.833400 13213 init.cc:377] Logged in from keytab as kudu/127.20.125.130@KRBTEST.COM (short username kudu)
I20260809 09:36:20.834374 13213 webserver.cc:460] Webserver started at http://127.20.125.130:40129/ using document root <none> and password file <none>
I20260809 09:36:20.835033 13213 fs_manager.cc:359] Metadata directory not provided
I20260809 09:36:20.835144 13213 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 09:36:20.839253 13213 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260809 09:36:20.841738 13236 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 09:36:20.843251 13213 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 09:36:20.843418 13213 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/minicluster-data/ts-1/data,/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/minicluster-data/ts-1/wal
uuid: "696cb7ca94004c299e1706b18b99498c"
format_stamp: "Formatted at 2026-08-09 09:35:38 on dist-test-slave-7kzw"
server_key: "e4ce44f94462a74aeb53e508794e8739"
server_key_iv: "98bda8f07cfb4c5d370c5a1017a92b61"
server_key_version: "encryptionkey@0"
I20260809 09:36:20.844046 13213 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 09:36:20.864763 13213 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 09:36:20.868537 13213 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 09:36:20.868799 13213 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 09:36:20.869519 13213 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 09:36:20.870507 13213 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 09:36:20.870589 13213 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 09:36:20.870671 13213 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 09:36:20.870736 13213 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 09:36:20.882018 13213 rpc_server.cc:307] RPC server started. Bound to: 127.20.125.130:33021
I20260809 09:36:20.882040 13406 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.125.130:33021 every 8 connection(s)
I20260809 09:36:20.883271 13213 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/minicluster-data/ts-1/data/info.pb
I20260809 09:36:20.887393 13411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:36:20.885530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:43909 (local address 127.20.125.130:45175)
0809 09:36:20.886050 (+ 520us) negotiation.cc:107] Waiting for socket to connect
0809 09:36:20.886633 (+ 583us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.190:43909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":316,"thread_start_us":128,"threads_started":1}
W20260809 09:36:20.887960 13408 heartbeater.cc:644] Failed to heartbeat to 127.20.125.190:43909 (0 consecutive failures): Network error: Failed to ping master at 127.20.125.190:43909: Client connection negotiation failed: client connection to 127.20.125.190:43909: connect: Connection refused (error 111)
I20260809 09:36:20.888728 13411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:36:20.888474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:43909 (local address 127.20.125.130:50441)
0809 09:36:20.888607 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0809 09:36:20.888665 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.190:43909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260809 09:36:20.889448 13411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:36:20.889227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:43909 (local address 127.20.125.130:39887)
0809 09:36:20.889339 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0809 09:36:20.889389 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.190:43909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260809 09:36:20.889734 13408 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260809 09:36:20.892802 20982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu as pid 13213
I20260809 09:36:20.893303 20982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
/tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.125.131:46207
--local_ip_for_outbound_sockets=127.20.125.131
--tserver_master_addrs=127.20.125.190:43909
--webserver_port=38713
--webserver_interface=127.20.125.131
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.131.keytab
--principal=kudu/127.20.125.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.125.148:36641
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 09:36:21.014601 13412 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 09:36:21.014855 13412 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 09:36:21.014896 13412 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 09:36:21.018836 13412 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 09:36:21.018890 13412 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 09:36:21.018986 13412 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.125.131
I20260809 09:36:21.024158 13412 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.125.148:36641
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.131.keytab
--principal=kudu/127.20.125.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.125.131:46207
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.125.131
--webserver_port=38713
--tserver_master_addrs=127.20.125.190:43909
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13412
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.125.131
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 08:38:55 UTC on ccycloud.quasar-njjleu.root.comops.site
build id 4575935
I20260809 09:36:21.025187 13412 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 09:36:21.026196 13412 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 09:36:21.035960 13421 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:36:21.036002 13422 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:36:21.036104 13412 server_base.cc:1034] running on GCE node
W20260809 09:36:21.036294 13424 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:36:21.036720 13412 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 09:36:21.037422 13412 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 09:36:21.038576 13412 hybrid_clock.cc:648] HybridClock initialized: now 1786268181038542 us; error 44 us; skew 500 ppm
Aug 09 09:36:21 dist-test-slave-7kzw krb5kdc[11335](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268181, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 09:36:21.041424 13412 init.cc:377] Logged in from keytab as kudu/127.20.125.131@KRBTEST.COM (short username kudu)
I20260809 09:36:21.042347 13412 webserver.cc:460] Webserver started at http://127.20.125.131:38713/ using document root <none> and password file <none>
I20260809 09:36:21.042943 13412 fs_manager.cc:359] Metadata directory not provided
I20260809 09:36:21.043012 13412 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 09:36:21.047209 13412 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260809 09:36:21.049934 13431 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 09:36:21.051234 13412 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 09:36:21.051375 13412 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/minicluster-data/ts-2/data,/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/minicluster-data/ts-2/wal
uuid: "e326d8d7643f49fba2572694f4a9a3c3"
format_stamp: "Formatted at 2026-08-09 09:35:38 on dist-test-slave-7kzw"
server_key: "b85b39b0443c81ffef29c4c085c8d970"
server_key_iv: "74e02f4580a8d8c897cee5a7de119270"
server_key_version: "encryptionkey@0"
I20260809 09:36:21.051919 13412 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 09:36:21.096688 13412 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 09:36:21.099912 13412 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 09:36:21.100122 13412 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 09:36:21.100739 13412 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 09:36:21.101591 13412 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 09:36:21.101666 13412 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 09:36:21.101740 13412 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 09:36:21.101794 13412 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 09:36:21.111479 13412 rpc_server.cc:307] RPC server started. Bound to: 127.20.125.131:46207
I20260809 09:36:21.111515 13601 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.125.131:46207 every 8 connection(s)
I20260809 09:36:21.112501 13412 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786267941944493-20982-0/minicluster-data/ts-2/data/info.pb
I20260809 09:36:21.116808 13604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:36:21.115188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:43909 (local address 127.20.125.131:36163)
0809 09:36:21.115688 (+ 500us) negotiation.cc:107] Waiting for socket to connect
0809 09:36:21.116133 (+ 445us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.190:43909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":294,"thread_start_us":126,"threads_started":1}
W20260809 09:36:21.117303 13602 heartbeater.cc:644] Failed to heartbeat to 127.20.125.190:43909 (0 consecutive failures): Network error: Failed to ping master at 127.20.125.190:43909: Client connection negotiation failed: client connection to 127.20.125.190:43909: connect: Connection refused (error 111)
I20260809 09:36:21.118014 13604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:36:21.117749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:43909 (local address 127.20.125.131:37249)
0809 09:36:21.117883 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0809 09:36:21.117943 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.190:43909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260809 09:36:21.118772 13604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:36:21.118531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:43909 (local address 127.20.125.131:43143)
0809 09:36:21.118665 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0809 09:36:21.118713 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.190:43909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
W20260809 09:36:21.118991 13602 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260809 09:36:21.122407 20982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu as pid 13412
I20260809 09:36:21.701678 13606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:36:21.701209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:43909 (local address 127.20.125.129:40855)
0809 09:36:21.701493 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0809 09:36:21.701554 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.190:43909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":85,"threads_started":1}
I20260809 09:36:21.890758 13607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:36:21.890270 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:43909 (local address 127.20.125.130:45187)
0809 09:36:21.890583 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0809 09:36:21.890639 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.190:43909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":72,"threads_started":1}
I20260809 09:36:22.119902 13609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:36:22.119518 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:43909 (local address 127.20.125.131:38433)
0809 09:36:22.119748 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0809 09:36:22.119805 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.190:43909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":70,"threads_started":1}
I20260809 09:36:22.169380 13610 raft_consensus.cc:482] T daa0a4a5135b46b7b854d8a99a6198b9 P 5ac1b284ab624af9bcc0235ebb0d2ead [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 09:36:22.169653 13610 raft_consensus.cc:504] T daa0a4a5135b46b7b854d8a99a6198b9 P 5ac1b284ab624af9bcc0235ebb0d2ead [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "696cb7ca94004c299e1706b18b99498c" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 33021 } } peers { permanent_uuid: "e326d8d7643f49fba2572694f4a9a3c3" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 46207 } } peers { permanent_uuid: "5ac1b284ab624af9bcc0235ebb0d2ead" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 42417 } }
I20260809 09:36:22.170949 13610 leader_election.cc:288] T daa0a4a5135b46b7b854d8a99a6198b9 P 5ac1b284ab624af9bcc0235ebb0d2ead [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 696cb7ca94004c299e1706b18b99498c (127.20.125.130:33021), e326d8d7643f49fba2572694f4a9a3c3 (127.20.125.131:46207)
Aug 09 09:36:22 dist-test-slave-7kzw krb5kdc[11335](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268180, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.129@KRBTEST.COM for kudu/127.20.125.131@KRBTEST.COM
Aug 09 09:36:22 dist-test-slave-7kzw krb5kdc[11335](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268180, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.129@KRBTEST.COM for kudu/127.20.125.130@KRBTEST.COM
I20260809 09:36:22.185398 13613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:36:22.171484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.129:54395 (local address 127.20.125.131:46207)
0809 09:36:22.171889 (+ 405us) server_negotiation.cc:206] Beginning negotiation
0809 09:36:22.171910 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0809 09:36:22.171941 (+ 31us) server_negotiation.cc:407] Connection header received
0809 09:36:22.172717 (+ 776us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:36:22.172726 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:36:22.173179 (+ 453us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:36:22.173519 (+ 340us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:36:22.174685 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:36:22.175657 (+ 972us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:36:22.177071 (+ 1414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:36:22.177551 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:36:22.177755 (+ 204us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:36:22.180569 (+ 2814us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:36:22.180579 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:36:22.180597 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:36:22.180646 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:36:22.182687 (+ 2041us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:36:22.183366 (+ 679us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:36:22.183379 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:36:22.183383 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:36:22.183495 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:36:22.183938 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:36:22.183940 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:36:22.183942 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:36:22.184309 (+ 367us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:36:22.184456 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0809 09:36:22.184979 (+ 523us) server_negotiation.cc:299] Negotiation successful
0809 09:36:22.185234 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":308,"thread_start_us":114,"threads_started":1}
I20260809 09:36:22.185385 13606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:36:22.171127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.130:33021 (local address 127.20.125.129:42509)
0809 09:36:22.171261 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0809 09:36:22.171278 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 09:36:22.171393 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:36:22.173577 (+ 2184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:36:22.173583 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:36:22.173979 (+ 396us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:36:22.174514 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:36:22.174529 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:36:22.175851 (+ 1322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:36:22.175856 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:36:22.176958 (+ 1102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:36:22.176968 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:36:22.177688 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:36:22.177692 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:36:22.177905 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:36:22.178683 (+ 778us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 09:36:22.178712 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:36:22.180672 (+ 1960us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 09:36:22.183008 (+ 2336us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:36:22.183011 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:36:22.183012 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:36:22.183245 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:36:22.183594 (+ 349us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:36:22.183598 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:36:22.183600 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:36:22.183771 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:36:22.184436 (+ 665us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 09:36:22.184442 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 09:36:22.184672 (+ 230us) client_negotiation.cc:769] Sending connection context
0809 09:36:22.184867 (+ 195us) client_negotiation.cc:240] Negotiation successful
0809 09:36:22.185192 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49,"mutex_wait_us":161,"spinlock_wait_cycles":15744}
I20260809 09:36:22.185467 13612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:36:22.171338 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.129:42509 (local address 127.20.125.130:33021)
0809 09:36:22.171685 (+ 347us) server_negotiation.cc:206] Beginning negotiation
0809 09:36:22.171712 (+ 27us) server_negotiation.cc:399] Waiting for connection header
0809 09:36:22.171759 (+ 47us) server_negotiation.cc:407] Connection header received
0809 09:36:22.172718 (+ 959us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:36:22.172726 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:36:22.173179 (+ 453us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:36:22.173519 (+ 340us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:36:22.174748 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:36:22.175706 (+ 958us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:36:22.177086 (+ 1380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:36:22.177553 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:36:22.177763 (+ 210us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:36:22.180896 (+ 3133us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:36:22.180901 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:36:22.180912 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:36:22.180939 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:36:22.182886 (+ 1947us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:36:22.183368 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:36:22.183374 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:36:22.183377 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:36:22.183491 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:36:22.183887 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:36:22.183893 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:36:22.183896 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:36:22.184311 (+ 415us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:36:22.184456 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0809 09:36:22.184980 (+ 524us) server_negotiation.cc:299] Negotiation successful
0809 09:36:22.185271 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":202,"thread_start_us":98,"threads_started":1}
I20260809 09:36:22.185490 13611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:36:22.171211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.131:46207 (local address 127.20.125.129:54395)
0809 09:36:22.171579 (+ 368us) negotiation.cc:107] Waiting for socket to connect
0809 09:36:22.171593 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 09:36:22.171734 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:36:22.174011 (+ 2277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:36:22.174013 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:36:22.174187 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:36:22.174443 (+ 256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:36:22.174458 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:36:22.175806 (+ 1348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:36:22.175809 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:36:22.176951 (+ 1142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:36:22.176965 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:36:22.177687 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:36:22.177691 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:36:22.177901 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:36:22.178715 (+ 814us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 09:36:22.178727 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:36:22.180317 (+ 1590us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 09:36:22.182810 (+ 2493us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:36:22.182818 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:36:22.182833 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:36:22.183241 (+ 408us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:36:22.183598 (+ 357us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:36:22.183601 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:36:22.183603 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:36:22.183798 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:36:22.184436 (+ 638us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 09:36:22.184442 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 09:36:22.184675 (+ 233us) client_negotiation.cc:769] Sending connection context
0809 09:36:22.184867 (+ 192us) client_negotiation.cc:240] Negotiation successful
0809 09:36:22.185281 (+ 414us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":308,"mutex_wait_us":303,"spinlock_wait_cycles":6272,"thread_start_us":91,"threads_started":1}
W20260809 09:36:22.188270 13052 leader_election.cc:341] T daa0a4a5135b46b7b854d8a99a6198b9 P 5ac1b284ab624af9bcc0235ebb0d2ead [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e326d8d7643f49fba2572694f4a9a3c3 (127.20.125.131:46207): Not found: Tablet not found: daa0a4a5135b46b7b854d8a99a6198b9
W20260809 09:36:22.188446 13049 leader_election.cc:341] T daa0a4a5135b46b7b854d8a99a6198b9 P 5ac1b284ab624af9bcc0235ebb0d2ead [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 696cb7ca94004c299e1706b18b99498c (127.20.125.130:33021): Not found: Tablet not found: daa0a4a5135b46b7b854d8a99a6198b9
I20260809 09:36:22.188546 13049 leader_election.cc:302] T daa0a4a5135b46b7b854d8a99a6198b9 P 5ac1b284ab624af9bcc0235ebb0d2ead [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5ac1b284ab624af9bcc0235ebb0d2ead; no voters: 696cb7ca94004c299e1706b18b99498c, e326d8d7643f49fba2572694f4a9a3c3
I20260809 09:36:22.188768 13610 raft_consensus.cc:2726] T daa0a4a5135b46b7b854d8a99a6198b9 P 5ac1b284ab624af9bcc0235ebb0d2ead [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 09:36:22.703044 13614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:36:22.702608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:43909 (local address 127.20.125.129:58397)
0809 09:36:22.702883 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0809 09:36:22.702940 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.190:43909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":81,"threads_started":1}
I20260809 09:36:22.892043 13616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:36:22.891626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:43909 (local address 127.20.125.130:48653)
0809 09:36:22.891892 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0809 09:36:22.891951 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.190:43909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":83,"threads_started":1}
I20260809 09:36:23.121201 13619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:36:23.120766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:43909 (local address 127.20.125.131:48647)
0809 09:36:23.121044 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0809 09:36:23.121109 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.190:43909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":103,"threads_started":1}
I20260809 09:36:23.704424 13620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:36:23.703983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:43909 (local address 127.20.125.129:37231)
0809 09:36:23.704265 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0809 09:36:23.704322 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.190:43909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":79,"threads_started":1}
I20260809 09:36:23.739260 13623 raft_consensus.cc:482] T daa0a4a5135b46b7b854d8a99a6198b9 P 5ac1b284ab624af9bcc0235ebb0d2ead [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 09:36:23.739426 13623 raft_consensus.cc:504] T daa0a4a5135b46b7b854d8a99a6198b9 P 5ac1b284ab624af9bcc0235ebb0d2ead [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "696cb7ca94004c299e1706b18b99498c" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 33021 } } peers { permanent_uuid: "e326d8d7643f49fba2572694f4a9a3c3" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 46207 } } peers { permanent_uuid: "5ac1b284ab624af9bcc0235ebb0d2ead" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 42417 } }
I20260809 09:36:23.739848 13623 leader_election.cc:288] T daa0a4a5135b46b7b854d8a99a6198b9 P 5ac1b284ab624af9bcc0235ebb0d2ead [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 696cb7ca94004c299e1706b18b99498c (127.20.125.130:33021), e326d8d7643f49fba2572694f4a9a3c3 (127.20.125.131:46207)
W20260809 09:36:23.740689 13049 leader_election.cc:341] T daa0a4a5135b46b7b854d8a99a6198b9 P 5ac1b284ab624af9bcc0235ebb0d2ead [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 696cb7ca94004c299e1706b18b99498c (127.20.125.130:33021): Not found: Tablet not found: daa0a4a5135b46b7b854d8a99a6198b9
W20260809 09:36:23.740826 13052 leader_election.cc:341] T daa0a4a5135b46b7b854d8a99a6198b9 P 5ac1b284ab624af9bcc0235ebb0d2ead [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e326d8d7643f49fba2572694f4a9a3c3 (127.20.125.131:46207): Not found: Tablet not found: daa0a4a5135b46b7b854d8a99a6198b9
I20260809 09:36:23.740929 13052 leader_election.cc:302] T daa0a4a5135b46b7b854d8a99a6198b9 P 5ac1b284ab624af9bcc0235ebb0d2ead [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5ac1b284ab624af9bcc0235ebb0d2ead; no voters: 696cb7ca94004c299e1706b18b99498c, e326d8d7643f49fba2572694f4a9a3c3
I20260809 09:36:23.741091 13623 raft_consensus.cc:2726] T daa0a4a5135b46b7b854d8a99a6198b9 P 5ac1b284ab624af9bcc0235ebb0d2ead [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 09:36:23.893219 13624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:36:23.892783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:43909 (local address 127.20.125.130:50917)
0809 09:36:23.893056 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0809 09:36:23.893114 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.190:43909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":79,"threads_started":1}
I20260809 09:36:24.122443 13626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:36:24.122017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:43909 (local address 127.20.125.131:32969)
0809 09:36:24.122293 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0809 09:36:24.122351 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.190:43909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":77,"threads_started":1}
I20260809 09:36:24.705891 13627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:36:24.705266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:43909 (local address 127.20.125.129:46669)
0809 09:36:24.705715 (+ 449us) negotiation.cc:107] Waiting for socket to connect
0809 09:36:24.705778 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.190:43909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":301,"thread_start_us":91,"threads_started":1}
I20260809 09:36:24.894639 13628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:36:24.894113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:43909 (local address 127.20.125.130:44247)
0809 09:36:24.894393 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0809 09:36:24.894451 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.190:43909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":98,"threads_started":1}
I20260809 09:36:25.123792 13629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:36:25.123310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:43909 (local address 127.20.125.131:46379)
0809 09:36:25.123625 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0809 09:36:25.123689 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.190:43909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":76,"threads_started":1}
I20260809 09:36:25.351126 13630 raft_consensus.cc:482] T daa0a4a5135b46b7b854d8a99a6198b9 P 5ac1b284ab624af9bcc0235ebb0d2ead [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 09:36:25.351317 13630 raft_consensus.cc:504] T daa0a4a5135b46b7b854d8a99a6198b9 P 5ac1b284ab624af9bcc0235ebb0d2ead [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "696cb7ca94004c299e1706b18b99498c" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 33021 } } peers { permanent_uuid: "e326d8d7643f49fba2572694f4a9a3c3" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 46207 } } peers { permanent_uuid: "5ac1b284ab624af9bcc0235ebb0d2ead" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 42417 } }
I20260809 09:36:25.351809 13630 leader_election.cc:288] T daa0a4a5135b46b7b854d8a99a6198b9 P 5ac1b284ab624af9bcc0235ebb0d2ead [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 696cb7ca94004c299e1706b18b99498c (127.20.125.130:33021), e326d8d7643f49fba2572694f4a9a3c3 (127.20.125.131:46207)
W20260809 09:36:25.352754 13049 leader_election.cc:341] T daa0a4a5135b46b7b854d8a99a6198b9 P 5ac1b284ab624af9bcc0235ebb0d2ead [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 696cb7ca94004c299e1706b18b99498c (127.20.125.130:33021): Not found: Tablet not found: daa0a4a5135b46b7b854d8a99a6198b9
W20260809 09:36:25.352845 13052 leader_election.cc:341] T daa0a4a5135b46b7b854d8a99a6198b9 P 5ac1b284ab624af9bcc0235ebb0d2ead [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e326d8d7643f49fba2572694f4a9a3c3 (127.20.125.131:46207): Not found: Tablet not found: daa0a4a5135b46b7b854d8a99a6198b9
I20260809 09:36:25.352910 13052 leader_election.cc:302] T daa0a4a5135b46b7b854d8a99a6198b9 P 5ac1b284ab624af9bcc0235ebb0d2ead [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5ac1b284ab624af9bcc0235ebb0d2ead; no voters: 696cb7ca94004c299e1706b18b99498c, e326d8d7643f49fba2572694f4a9a3c3
I20260809 09:36:25.353096 13630 raft_consensus.cc:2726] T daa0a4a5135b46b7b854d8a99a6198b9 P 5ac1b284ab624af9bcc0235ebb0d2ead [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 09:36:25.707573 13631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:36:25.706980 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:43909 (local address 127.20.125.129:39585)
0809 09:36:25.707400 (+ 420us) negotiation.cc:107] Waiting for socket to connect
0809 09:36:25.707465 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.190:43909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":82,"threads_started":1}
I20260809 09:36:25.895984 13632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:36:25.895490 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:43909 (local address 127.20.125.130:46009)
0809 09:36:25.895810 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0809 09:36:25.895886 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.190:43909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":236,"thread_start_us":89,"threads_started":1}
I20260809 09:36:26.125236 13634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:36:26.124744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:43909 (local address 127.20.125.131:56323)
0809 09:36:26.125040 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0809 09:36:26.125110 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.190:43909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":86,"threads_started":1}
I20260809 09:36:26.708856 13636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:36:26.708402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:43909 (local address 127.20.125.129:45549)
0809 09:36:26.708694 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0809 09:36:26.708751 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.190:43909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":84,"threads_started":1}
I20260809 09:36:26.897524 13637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:36:26.897057 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:43909 (local address 127.20.125.130:58483)
0809 09:36:26.897355 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0809 09:36:26.897417 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.190:43909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":100,"threads_started":1}
I20260809 09:36:27.126613 13640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:36:27.126091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:43909 (local address 127.20.125.131:36945)
0809 09:36:27.126386 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0809 09:36:27.126450 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.190:43909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":75,"threads_started":1}
I20260809 09:36:27.379249 13641 raft_consensus.cc:482] T daa0a4a5135b46b7b854d8a99a6198b9 P 5ac1b284ab624af9bcc0235ebb0d2ead [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 09:36:27.379388 13641 raft_consensus.cc:504] T daa0a4a5135b46b7b854d8a99a6198b9 P 5ac1b284ab624af9bcc0235ebb0d2ead [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "696cb7ca94004c299e1706b18b99498c" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 33021 } } peers { permanent_uuid: "e326d8d7643f49fba2572694f4a9a3c3" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 46207 } } peers { permanent_uuid: "5ac1b284ab624af9bcc0235ebb0d2ead" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 42417 } }
I20260809 09:36:27.379889 13641 leader_election.cc:288] T daa0a4a5135b46b7b854d8a99a6198b9 P 5ac1b284ab624af9bcc0235ebb0d2ead [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 696cb7ca94004c299e1706b18b99498c (127.20.125.130:33021), e326d8d7643f49fba2572694f4a9a3c3 (127.20.125.131:46207)
W20260809 09:36:27.380697 13052 leader_election.cc:341] T daa0a4a5135b46b7b854d8a99a6198b9 P 5ac1b284ab624af9bcc0235ebb0d2ead [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e326d8d7643f49fba2572694f4a9a3c3 (127.20.125.131:46207): Not found: Tablet not found: daa0a4a5135b46b7b854d8a99a6198b9
W20260809 09:36:27.380787 13049 leader_election.cc:341] T daa0a4a5135b46b7b854d8a99a6198b9 P 5ac1b284ab624af9bcc0235ebb0d2ead [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 696cb7ca94004c299e1706b18b99498c (127.20.125.130:33021): Not found: Tablet not found: daa0a4a5135b46b7b854d8a99a6198b9
I20260809 09:36:27.380851 13049 leader_election.cc:302] T daa0a4a5135b46b7b854d8a99a6198b9 P 5ac1b284ab624af9bcc0235ebb0d2ead [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5ac1b284ab624af9bcc0235ebb0d2ead; no voters: 696cb7ca94004c299e1706b18b99498c, e326d8d7643f49fba2572694f4a9a3c3
I20260809 09:36:27.380986 13641 raft_consensus.cc:2726] T daa0a4a5135b46b7b854d8a99a6198b9 P 5ac1b284ab624af9bcc0235ebb0d2ead [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 09:36:27.710268 13643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:36:27.709794 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:43909 (local address 127.20.125.129:37879)
0809 09:36:27.710081 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0809 09:36:27.710157 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.190:43909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":79,"threads_started":1}
I20260809 09:36:27.898989 13644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:36:27.898379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:43909 (local address 127.20.125.130:46091)
0809 09:36:27.898822 (+ 443us) negotiation.cc:107] Waiting for socket to connect
0809 09:36:27.898884 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.190:43909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":364,"thread_start_us":144,"threads_started":1}
I20260809 09:36:28.127835 13645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:36:28.127364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:43909 (local address 127.20.125.131:59585)
0809 09:36:28.127682 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0809 09:36:28.127738 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.190:43909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":238,"thread_start_us":75,"threads_started":1}
I20260809 09:36:28.711634 13646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:36:28.711175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:43909 (local address 127.20.125.129:33181)
0809 09:36:28.711456 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0809 09:36:28.711525 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.190:43909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":85,"threads_started":1}
I20260809 09:36:28.900403 13647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:36:28.899869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:43909 (local address 127.20.125.130:33329)
0809 09:36:28.900234 (+ 365us) negotiation.cc:107] Waiting for socket to connect
0809 09:36:28.900297 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.190:43909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":282,"thread_start_us":148,"threads_started":1}
I20260809 09:36:29.129140 13650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:36:29.128690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:43909 (local address 127.20.125.131:34253)
0809 09:36:29.128972 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0809 09:36:29.129030 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.190:43909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":108,"threads_started":1}
I20260809 09:36:29.712857 13651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:36:29.712415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:43909 (local address 127.20.125.129:47553)
0809 09:36:29.712688 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0809 09:36:29.712753 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.190:43909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":73,"threads_started":1}
I20260809 09:36:29.901734 13654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:36:29.901236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:43909 (local address 127.20.125.130:45497)
0809 09:36:29.901539 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0809 09:36:29.901631 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.190:43909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":86,"threads_started":1}
I20260809 09:36:30.130381 13656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:36:30.129938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:43909 (local address 127.20.125.131:34835)
0809 09:36:30.130216 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0809 09:36:30.130278 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.190:43909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":77,"threads_started":1}
I20260809 09:36:30.714375 13657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:36:30.713843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:43909 (local address 127.20.125.129:54815)
0809 09:36:30.714156 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0809 09:36:30.714247 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.190:43909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":76,"threads_started":1}
I20260809 09:36:30.903143 13658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:36:30.902716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:43909 (local address 127.20.125.130:56853)
0809 09:36:30.902981 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0809 09:36:30.903040 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.190:43909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":81,"threads_started":1}
I20260809 09:36:31.131770 13659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:36:31.131274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:43909 (local address 127.20.125.131:33513)
0809 09:36:31.131602 (+ 328us) negotiation.cc:107] Waiting for socket to connect
0809 09:36:31.131665 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.190:43909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":103,"threads_started":1}
I20260809 09:36:31.715682 13661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:36:31.715236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:43909 (local address 127.20.125.129:54465)
0809 09:36:31.715507 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0809 09:36:31.715572 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.190:43909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":72,"threads_started":1}
I20260809 09:36:31.904343 13662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:36:31.903929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:43909 (local address 127.20.125.130:49167)
0809 09:36:31.904181 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0809 09:36:31.904238 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.190:43909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":78,"threads_started":1}
I20260809 09:36:32.133136 13664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:36:32.132699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:43909 (local address 127.20.125.131:38571)
0809 09:36:32.132983 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0809 09:36:32.133037 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.190:43909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":113,"threads_started":1}
I20260809 09:36:32.717154 13667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:36:32.716474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:43909 (local address 127.20.125.129:54761)
0809 09:36:32.716963 (+ 489us) negotiation.cc:107] Waiting for socket to connect
0809 09:36:32.717031 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.190:43909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":372,"thread_start_us":118,"threads_started":1}
I20260809 09:36:32.905661 13668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:36:32.905186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:43909 (local address 127.20.125.130:52521)
0809 09:36:32.905474 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0809 09:36:32.905546 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.190:43909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":79,"threads_started":1}
I20260809 09:36:33.134397 13672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:36:33.133954 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:43909 (local address 127.20.125.131:50513)
0809 09:36:33.134242 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0809 09:36:33.134299 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.190:43909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":112,"threads_started":1}
I20260809 09:36:33.718408 13673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:36:33.717949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:43909 (local address 127.20.125.129:39539)
0809 09:36:33.718249 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0809 09:36:33.718305 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.190:43909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":111,"threads_started":1}
I20260809 09:36:33.907197 13674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:36:33.906646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:43909 (local address 127.20.125.130:55201)
0809 09:36:33.907016 (+ 370us) negotiation.cc:107] Waiting for socket to connect
0809 09:36:33.907082 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.190:43909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":288,"thread_start_us":143,"threads_started":1}
I20260809 09:36:34.135608 13676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:36:34.135207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:43909 (local address 127.20.125.131:44949)
0809 09:36:34.135451 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0809 09:36:34.135508 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.190:43909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":76,"threads_started":1}
I20260809 09:36:34.719645 13677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:36:34.719226 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:43909 (local address 127.20.125.129:58105)
0809 09:36:34.719486 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0809 09:36:34.719542 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.190:43909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":96,"threads_started":1}
I20260809 09:36:34.769506 13678 raft_consensus.cc:482] T daa0a4a5135b46b7b854d8a99a6198b9 P 5ac1b284ab624af9bcc0235ebb0d2ead [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 09:36:34.769654 13678 raft_consensus.cc:504] T daa0a4a5135b46b7b854d8a99a6198b9 P 5ac1b284ab624af9bcc0235ebb0d2ead [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "696cb7ca94004c299e1706b18b99498c" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 33021 } } peers { permanent_uuid: "e326d8d7643f49fba2572694f4a9a3c3" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 46207 } } peers { permanent_uuid: "5ac1b284ab624af9bcc0235ebb0d2ead" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 42417 } }
I20260809 09:36:34.770216 13678 leader_election.cc:288] T daa0a4a5135b46b7b854d8a99a6198b9 P 5ac1b284ab624af9bcc0235ebb0d2ead [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 696cb7ca94004c299e1706b18b99498c (127.20.125.130:33021), e326d8d7643f49fba2572694f4a9a3c3 (127.20.125.131:46207)
W20260809 09:36:34.771139 13049 leader_election.cc:341] T daa0a4a5135b46b7b854d8a99a6198b9 P 5ac1b284ab624af9bcc0235ebb0d2ead [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 696cb7ca94004c299e1706b18b99498c (127.20.125.130:33021): Not found: Tablet not found: daa0a4a5135b46b7b854d8a99a6198b9
W20260809 09:36:34.771286 13052 leader_election.cc:341] T daa0a4a5135b46b7b854d8a99a6198b9 P 5ac1b284ab624af9bcc0235ebb0d2ead [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e326d8d7643f49fba2572694f4a9a3c3 (127.20.125.131:46207): Not found: Tablet not found: daa0a4a5135b46b7b854d8a99a6198b9
I20260809 09:36:34.771394 13052 leader_election.cc:302] T daa0a4a5135b46b7b854d8a99a6198b9 P 5ac1b284ab624af9bcc0235ebb0d2ead [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5ac1b284ab624af9bcc0235ebb0d2ead; no voters: 696cb7ca94004c299e1706b18b99498c, e326d8d7643f49fba2572694f4a9a3c3
I20260809 09:36:34.771597 13678 raft_consensus.cc:2726] T daa0a4a5135b46b7b854d8a99a6198b9 P 5ac1b284ab624af9bcc0235ebb0d2ead [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 09:36:34.908573 13679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:36:34.908117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:43909 (local address 127.20.125.130:40705)
0809 09:36:34.908417 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0809 09:36:34.908475 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.190:43909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":100,"threads_started":1}
I20260809 09:36:35.137023 13680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:36:35.136529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:43909 (local address 127.20.125.131:34177)
0809 09:36:35.136856 (+ 327us) negotiation.cc:107] Waiting for socket to connect
0809 09:36:35.136920 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.190:43909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":245,"thread_start_us":113,"threads_started":1}
I20260809 09:36:35.721264 13681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:36:35.720855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:43909 (local address 127.20.125.129:38901)
0809 09:36:35.721106 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0809 09:36:35.721160 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.190:43909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":74,"threads_started":1}
I20260809 09:36:35.910038 13682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:36:35.909571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:43909 (local address 127.20.125.130:58965)
0809 09:36:35.909861 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0809 09:36:35.909929 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.190:43909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":105,"threads_started":1}
I20260809 09:36:36.128711 20982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu with pid 12971
I20260809 09:36:36.141510 13684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:36:36.138437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:43909 (local address 127.20.125.131:36641)
0809 09:36:36.141288 (+ 2851us) negotiation.cc:107] Waiting for socket to connect
0809 09:36:36.141370 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.190:43909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2732,"thread_start_us":579,"threads_started":1}
I20260809 09:36:36.148267 20982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu with pid 13213
I20260809 09:36:36.162988 20982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu with pid 13412
2026-08-09T09:36:36Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (59739 ms)
[----------] 30 tests from SecurityITest (254219 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786267941944493-20982-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 09:36:36 dist-test-slave-7kzw krb5kdc[13687](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 09:36:36 dist-test-slave-7kzw krb5kdc[13687](info): set up 2 sockets
Aug 09 09:36:36 dist-test-slave-7kzw krb5kdc[13687](info): commencing operation
krb5kdc: starting...
W20260809 09:36:37.407642 20982 mini_kdc.cc:121] Time spent starting KDC: real 1.215s user 0.003s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786267941944493-20982-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786267941944493-20982-0/krb5kdc/test-user.keytab.
Aug 09 09:36:37 dist-test-slave-7kzw krb5kdc[13687](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268197, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T09:36:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T09:36:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.125.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.125.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.125.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.190.keytab.
Entry for principal kudu/127.20.125.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.190.keytab.
I20260809 09:36:37.539435 20982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
/tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786267941944493-20982-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786267941944493-20982-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786267941944493-20982-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786267941944493-20982-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.125.190:42053
--webserver_interface=127.20.125.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.190.keytab
--principal=kudu/127.20.125.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.125.148:35723
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.125.190:42053
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786267941944493-20982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786267941944493-20982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786267941944493-20982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 09:36:37.651764 13705 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 09:36:37.652092 13705 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 09:36:37.652171 13705 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 09:36:37.656160 13705 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 09:36:37.656268 13705 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 09:36:37.656308 13705 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 09:36:37.656347 13705 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 09:36:37.656384 13705 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 09:36:37.662393 13705 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.125.148:35723
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786267941944493-20982-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786267941944493-20982-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.125.190:42053
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.190.keytab
--principal=kudu/127.20.125.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.125.190:42053
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786267941944493-20982-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.125.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13705
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786267941944493-20982-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 08:38:55 UTC on ccycloud.quasar-njjleu.root.comops.site
build id 4575935
I20260809 09:36:37.663600 13705 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 09:36:37.664710 13705 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 09:36:37.676191 13705 server_base.cc:1034] running on GCE node
W20260809 09:36:37.676165 13713 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:36:37.676205 13715 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:36:37.676483 13711 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:36:37.677258 13705 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 09:36:37.678784 13705 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 09:36:37.680017 13705 hybrid_clock.cc:648] HybridClock initialized: now 1786268197679994 us; error 34 us; skew 500 ppm
Aug 09 09:36:37 dist-test-slave-7kzw krb5kdc[13687](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268197, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 09:36:37.683236 13705 init.cc:377] Logged in from keytab as kudu/127.20.125.190@KRBTEST.COM (short username kudu)
I20260809 09:36:37.684283 13705 webserver.cc:460] Webserver started at http://127.20.125.190:33107/ using document root <none> and password file <none>
I20260809 09:36:37.684926 13705 fs_manager.cc:359] Metadata directory not provided
I20260809 09:36:37.685034 13705 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 09:36:37.685304 13705 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 09:36:37.687693 13705 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786267941944493-20982-0/minicluster-data/master-0/data/instance:
uuid: "40ae5ae7dd2e4ef8b86807697ff48b80"
format_stamp: "Formatted at 2026-08-09 09:36:37 on dist-test-slave-7kzw"
server_key: "a287b8bcc64add592877c47097c08167"
server_key_iv: "71376c7b90c7c9bda704780afdf1c4b4"
server_key_version: "encryptionkey@0"
I20260809 09:36:37.688325 13705 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786267941944493-20982-0/minicluster-data/master-0/wal/instance:
uuid: "40ae5ae7dd2e4ef8b86807697ff48b80"
format_stamp: "Formatted at 2026-08-09 09:36:37 on dist-test-slave-7kzw"
server_key: "a287b8bcc64add592877c47097c08167"
server_key_iv: "71376c7b90c7c9bda704780afdf1c4b4"
server_key_version: "encryptionkey@0"
I20260809 09:36:37.692498 13705 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 09:36:37.695176 13723 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 09:36:37.696506 13705 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 09:36:37.696686 13705 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786267941944493-20982-0/minicluster-data/master-0/data,/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786267941944493-20982-0/minicluster-data/master-0/wal
uuid: "40ae5ae7dd2e4ef8b86807697ff48b80"
format_stamp: "Formatted at 2026-08-09 09:36:37 on dist-test-slave-7kzw"
server_key: "a287b8bcc64add592877c47097c08167"
server_key_iv: "71376c7b90c7c9bda704780afdf1c4b4"
server_key_version: "encryptionkey@0"
I20260809 09:36:37.696825 13705 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786267941944493-20982-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786267941944493-20982-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786267941944493-20982-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 09:36:37.713465 13705 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 09:36:37.716859 13705 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 09:36:37.717106 13705 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 09:36:37.725179 13705 rpc_server.cc:307] RPC server started. Bound to: 127.20.125.190:42053
I20260809 09:36:37.725292 13796 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.125.190:42053 every 8 connection(s)
I20260809 09:36:37.726428 13705 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786267941944493-20982-0/minicluster-data/master-0/data/info.pb
I20260809 09:36:37.729545 13797 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 09:36:37.735636 13797 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 40ae5ae7dd2e4ef8b86807697ff48b80: Bootstrap starting.
I20260809 09:36:37.736555 20982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu as pid 13705
I20260809 09:36:37.736694 20982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786267941944493-20982-0/minicluster-data/master-0/wal/instance
I20260809 09:36:37.737043 20982 external_mini_cluster.cc:1428] Setting key 88ad9296ec60f773025dee5abdeaab4d
I20260809 09:36:37.739035 13797 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 40ae5ae7dd2e4ef8b86807697ff48b80: Neither blocks nor log segments found. Creating new log.
I20260809 09:36:37.739866 13797 log.cc:824] T 00000000000000000000000000000000 P 40ae5ae7dd2e4ef8b86807697ff48b80: Log is configured to *not* fsync() on all Append() calls
I20260809 09:36:37.741881 13797 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 40ae5ae7dd2e4ef8b86807697ff48b80: No bootstrap required, opened a new log
I20260809 09:36:37.745618 13797 raft_consensus.cc:348] T 00000000000000000000000000000000 P 40ae5ae7dd2e4ef8b86807697ff48b80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40ae5ae7dd2e4ef8b86807697ff48b80" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 42053 } }
I20260809 09:36:37.745889 13797 raft_consensus.cc:374] T 00000000000000000000000000000000 P 40ae5ae7dd2e4ef8b86807697ff48b80 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 09:36:37.745992 13797 raft_consensus.cc:729] T 00000000000000000000000000000000 P 40ae5ae7dd2e4ef8b86807697ff48b80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 40ae5ae7dd2e4ef8b86807697ff48b80, State: Initialized, Role: FOLLOWER
I20260809 09:36:37.746515 13797 consensus_queue.cc:260] T 00000000000000000000000000000000 P 40ae5ae7dd2e4ef8b86807697ff48b80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40ae5ae7dd2e4ef8b86807697ff48b80" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 42053 } }
I20260809 09:36:37.746706 13797 raft_consensus.cc:388] T 00000000000000000000000000000000 P 40ae5ae7dd2e4ef8b86807697ff48b80 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
Aug 09 09:36:37 dist-test-slave-7kzw krb5kdc[13687](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268197, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.125.190@KRBTEST.COM
I20260809 09:36:37.746803 13797 raft_consensus.cc:482] T 00000000000000000000000000000000 P 40ae5ae7dd2e4ef8b86807697ff48b80 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 09:36:37.746913 13797 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 40ae5ae7dd2e4ef8b86807697ff48b80 [term 0 FOLLOWER]: Advancing to term 1
I20260809 09:36:37.748136 13797 raft_consensus.cc:504] T 00000000000000000000000000000000 P 40ae5ae7dd2e4ef8b86807697ff48b80 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40ae5ae7dd2e4ef8b86807697ff48b80" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 42053 } }
I20260809 09:36:37.748510 13797 leader_election.cc:302] T 00000000000000000000000000000000 P 40ae5ae7dd2e4ef8b86807697ff48b80 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 40ae5ae7dd2e4ef8b86807697ff48b80; no voters:
I20260809 09:36:37.748919 13797 leader_election.cc:288] T 00000000000000000000000000000000 P 40ae5ae7dd2e4ef8b86807697ff48b80 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 09:36:37.749032 13803 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 40ae5ae7dd2e4ef8b86807697ff48b80 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 09:36:37.749297 13803 raft_consensus.cc:686] T 00000000000000000000000000000000 P 40ae5ae7dd2e4ef8b86807697ff48b80 [term 1 LEADER]: Becoming Leader. State: Replica: 40ae5ae7dd2e4ef8b86807697ff48b80, State: Running, Role: LEADER
I20260809 09:36:37.749687 13803 consensus_queue.cc:237] T 00000000000000000000000000000000 P 40ae5ae7dd2e4ef8b86807697ff48b80 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40ae5ae7dd2e4ef8b86807697ff48b80" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 42053 } }
I20260809 09:36:37.750438 13797 sys_catalog.cc:564] T 00000000000000000000000000000000 P 40ae5ae7dd2e4ef8b86807697ff48b80 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 09:36:37.750999 13808 sys_catalog.cc:455] T 00000000000000000000000000000000 P 40ae5ae7dd2e4ef8b86807697ff48b80 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 40ae5ae7dd2e4ef8b86807697ff48b80. Latest consensus state: current_term: 1 leader_uuid: "40ae5ae7dd2e4ef8b86807697ff48b80" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40ae5ae7dd2e4ef8b86807697ff48b80" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 42053 } } }
I20260809 09:36:37.751199 13808 sys_catalog.cc:458] T 00000000000000000000000000000000 P 40ae5ae7dd2e4ef8b86807697ff48b80 [sys.catalog]: This master's current role is: LEADER
I20260809 09:36:37.751638 13812 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 09:36:37.754056 13800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:36:37.738423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36670 (local address 127.20.125.190:42053)
0809 09:36:37.738965 (+ 542us) server_negotiation.cc:206] Beginning negotiation
0809 09:36:37.738983 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0809 09:36:37.739022 (+ 39us) server_negotiation.cc:407] Connection header received
0809 09:36:37.739765 (+ 743us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:36:37.739784 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:36:37.740209 (+ 425us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:36:37.740576 (+ 367us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:36:37.741511 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:36:37.742731 (+ 1220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:36:37.743682 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:36:37.744200 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:36:37.744388 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:36:37.747544 (+ 3156us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:36:37.747550 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:36:37.747564 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:36:37.747593 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:36:37.749939 (+ 2346us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:36:37.750546 (+ 607us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:36:37.750552 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:36:37.750557 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:36:37.750711 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:36:37.751089 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:36:37.751093 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:36:37.751095 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:36:37.751705 (+ 610us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:36:37.751922 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0809 09:36:37.752809 (+ 887us) server_negotiation.cc:299] Negotiation successful
0809 09:36:37.753141 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":383,"thread_start_us":169,"threads_started":1}
I20260809 09:36:37.754971 13807 sys_catalog.cc:455] T 00000000000000000000000000000000 P 40ae5ae7dd2e4ef8b86807697ff48b80 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "40ae5ae7dd2e4ef8b86807697ff48b80" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40ae5ae7dd2e4ef8b86807697ff48b80" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 42053 } } }
I20260809 09:36:37.755129 13807 sys_catalog.cc:458] T 00000000000000000000000000000000 P 40ae5ae7dd2e4ef8b86807697ff48b80 [sys.catalog]: This master's current role is: LEADER
I20260809 09:36:37.756820 13812 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 09:36:37.761981 13812 catalog_manager.cc:1347] Generated new cluster ID: e2d84f0686a1411aa138756d2cebef54
I20260809 09:36:37.762050 13812 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 09:36:37.773309 13812 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 09:36:37.774191 13812 catalog_manager.cc:1504] Loading token signing keys...
I20260809 09:36:37.787338 13812 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 40ae5ae7dd2e4ef8b86807697ff48b80: Generated new TSK 0
I20260809 09:36:37.788072 13812 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.20.125.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.125.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.125.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.129.keytab.
Entry for principal kudu/127.20.125.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.129.keytab.
I20260809 09:36:37.825215 20982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
/tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786267941944493-20982-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786267941944493-20982-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786267941944493-20982-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786267941944493-20982-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.125.129:0
--local_ip_for_outbound_sockets=127.20.125.129
--webserver_interface=127.20.125.129
--webserver_port=0
--tserver_master_addrs=127.20.125.190:42053
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.129.keytab
--principal=kudu/127.20.125.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.125.148:35723
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786267941944493-20982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786267941944493-20982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786267941944493-20982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 09:36:37.940877 13843 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 09:36:37.941133 13843 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 09:36:37.941174 13843 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 09:36:37.945168 13843 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 09:36:37.945227 13843 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 09:36:37.945283 13843 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.125.129
I20260809 09:36:37.950678 13843 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.125.148:35723
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786267941944493-20982-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786267941944493-20982-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.129.keytab
--principal=kudu/127.20.125.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.125.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786267941944493-20982-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.125.129
--webserver_port=0
--tserver_master_addrs=127.20.125.190:42053
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13843
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.125.129
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786267941944493-20982-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 08:38:55 UTC on ccycloud.quasar-njjleu.root.comops.site
build id 4575935
I20260809 09:36:37.951929 13843 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 09:36:37.953024 13843 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 09:36:37.962738 13852 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:36:37.962777 13843 server_base.cc:1034] running on GCE node
W20260809 09:36:37.962738 13850 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:36:37.963061 13855 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:36:37.963575 13843 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 09:36:37.964262 13843 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 09:36:37.965430 13843 hybrid_clock.cc:648] HybridClock initialized: now 1786268197965415 us; error 27 us; skew 500 ppm
Aug 09 09:36:37 dist-test-slave-7kzw krb5kdc[13687](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268197, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 09:36:37.968564 13843 init.cc:377] Logged in from keytab as kudu/127.20.125.129@KRBTEST.COM (short username kudu)
I20260809 09:36:37.969494 13843 webserver.cc:460] Webserver started at http://127.20.125.129:33761/ using document root <none> and password file <none>
I20260809 09:36:37.970099 13843 fs_manager.cc:359] Metadata directory not provided
I20260809 09:36:37.970206 13843 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 09:36:37.970504 13843 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 09:36:37.972704 13843 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786267941944493-20982-0/minicluster-data/ts-0/data/instance:
uuid: "74cb73ba683f49a18f0b3af5f5ed9ebd"
format_stamp: "Formatted at 2026-08-09 09:36:37 on dist-test-slave-7kzw"
server_key: "9fba3e086f4655c77153956dab157c1f"
server_key_iv: "f5a3eaf4a9c4045a2faedd83e4ca6308"
server_key_version: "encryptionkey@0"
I20260809 09:36:37.973325 13843 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786267941944493-20982-0/minicluster-data/ts-0/wal/instance:
uuid: "74cb73ba683f49a18f0b3af5f5ed9ebd"
format_stamp: "Formatted at 2026-08-09 09:36:37 on dist-test-slave-7kzw"
server_key: "9fba3e086f4655c77153956dab157c1f"
server_key_iv: "f5a3eaf4a9c4045a2faedd83e4ca6308"
server_key_version: "encryptionkey@0"
I20260809 09:36:37.977273 13843 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260809 09:36:37.979888 13864 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 09:36:37.981107 13843 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 09:36:37.981303 13843 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786267941944493-20982-0/minicluster-data/ts-0/data,/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786267941944493-20982-0/minicluster-data/ts-0/wal
uuid: "74cb73ba683f49a18f0b3af5f5ed9ebd"
format_stamp: "Formatted at 2026-08-09 09:36:37 on dist-test-slave-7kzw"
server_key: "9fba3e086f4655c77153956dab157c1f"
server_key_iv: "f5a3eaf4a9c4045a2faedd83e4ca6308"
server_key_version: "encryptionkey@0"
I20260809 09:36:37.981420 13843 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786267941944493-20982-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786267941944493-20982-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786267941944493-20982-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 09:36:37.994427 13843 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 09:36:37.997951 13843 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 09:36:37.998179 13843 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 09:36:37.998870 13843 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 09:36:37.999747 13843 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 09:36:37.999821 13843 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 09:36:37.999899 13843 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 09:36:37.999954 13843 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 09:36:38.009210 13843 rpc_server.cc:307] RPC server started. Bound to: 127.20.125.129:37127
I20260809 09:36:38.009249 14027 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.125.129:37127 every 8 connection(s)
I20260809 09:36:38.010118 13843 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786267941944493-20982-0/minicluster-data/ts-0/data/info.pb
I20260809 09:36:38.013092 20982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu as pid 13843
I20260809 09:36:38.013237 20982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786267941944493-20982-0/minicluster-data/ts-0/wal/instance
I20260809 09:36:38.013556 20982 external_mini_cluster.cc:1428] Setting key b5901422456c7fed5b79bf47813f5635
Aug 09 09:36:38 dist-test-slave-7kzw krb5kdc[13687](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268197, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.129@KRBTEST.COM for kudu/127.20.125.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.20.125.130@KRBTEST.COM; defaulting to no policy
I20260809 09:36:38.025209 13800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:36:38.012065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.129:37939 (local address 127.20.125.190:42053)
0809 09:36:38.012231 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0809 09:36:38.012235 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 09:36:38.013164 (+ 929us) server_negotiation.cc:407] Connection header received
0809 09:36:38.014951 (+ 1787us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:36:38.014957 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:36:38.015015 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:36:38.015101 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:36:38.016304 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:36:38.017141 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:36:38.018302 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:36:38.018616 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:36:38.018773 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:36:38.021347 (+ 2574us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:36:38.021351 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:36:38.021354 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:36:38.021386 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:36:38.022976 (+ 1590us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:36:38.023503 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:36:38.023509 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:36:38.023514 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:36:38.023602 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:36:38.024018 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:36:38.024023 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:36:38.024025 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:36:38.024219 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:36:38.024330 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0809 09:36:38.024875 (+ 545us) server_negotiation.cc:299] Negotiation successful
0809 09:36:38.025023 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260809 09:36:38.026192 14033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:36:38.012501 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:42053 (local address 127.20.125.129:37939)
0809 09:36:38.013000 (+ 499us) negotiation.cc:107] Waiting for socket to connect
0809 09:36:38.013044 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 09:36:38.014696 (+ 1652us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:36:38.015315 (+ 619us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:36:38.015322 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:36:38.015679 (+ 357us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:36:38.016141 (+ 462us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:36:38.016153 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:36:38.017298 (+ 1145us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:36:38.017300 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:36:38.018161 (+ 861us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:36:38.018170 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:36:38.018810 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:36:38.018814 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:36:38.018985 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:36:38.019631 (+ 646us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 09:36:38.019647 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:36:38.021180 (+ 1533us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 09:36:38.023097 (+ 1917us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:36:38.023103 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:36:38.023119 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:36:38.023396 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:36:38.023764 (+ 368us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:36:38.023767 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:36:38.023769 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:36:38.023919 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:36:38.024327 (+ 408us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 09:36:38.024335 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 09:36:38.024633 (+ 298us) client_negotiation.cc:769] Sending connection context
0809 09:36:38.024831 (+ 198us) client_negotiation.cc:240] Negotiation successful
0809 09:36:38.025164 (+ 333us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":310,"thread_start_us":117,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.125.130@KRBTEST.COM" created.
I20260809 09:36:38.027714 14030 heartbeater.cc:347] Connected to a master server at 127.20.125.190:42053
I20260809 09:36:38.027982 14030 heartbeater.cc:464] Registering TS with master...
I20260809 09:36:38.028615 14030 heartbeater.cc:511] Master 127.20.125.190:42053 requested a full tablet report, sending...
I20260809 09:36:38.030418 13750 ts_manager.cc:194] Registered new tserver with Master: 74cb73ba683f49a18f0b3af5f5ed9ebd (127.20.125.129:37127)
I20260809 09:36:38.031616 13750 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.125.129@KRBTEST.COM'} at 127.20.125.129:37939
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.125.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.130.keytab.
Entry for principal kudu/127.20.125.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.130.keytab.
I20260809 09:36:38.045426 20982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
/tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786267941944493-20982-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786267941944493-20982-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786267941944493-20982-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786267941944493-20982-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.125.130:0
--local_ip_for_outbound_sockets=127.20.125.130
--webserver_interface=127.20.125.130
--webserver_port=0
--tserver_master_addrs=127.20.125.190:42053
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.130.keytab
--principal=kudu/127.20.125.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.125.148:35723
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786267941944493-20982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786267941944493-20982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786267941944493-20982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 09:36:38.163158 14037 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 09:36:38.163403 14037 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 09:36:38.163444 14037 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 09:36:38.167446 14037 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 09:36:38.167501 14037 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 09:36:38.167593 14037 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.125.130
I20260809 09:36:38.172803 14037 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.125.148:35723
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786267941944493-20982-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786267941944493-20982-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.130.keytab
--principal=kudu/127.20.125.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.125.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786267941944493-20982-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.125.130
--webserver_port=0
--tserver_master_addrs=127.20.125.190:42053
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14037
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.125.130
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786267941944493-20982-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 08:38:55 UTC on ccycloud.quasar-njjleu.root.comops.site
build id 4575935
I20260809 09:36:38.173817 14037 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 09:36:38.174916 14037 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 09:36:38.185503 14043 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:36:38.185546 14044 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:36:38.185667 14048 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:36:38.185796 14037 server_base.cc:1034] running on GCE node
I20260809 09:36:38.186268 14037 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 09:36:38.186945 14037 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 09:36:38.188148 14037 hybrid_clock.cc:648] HybridClock initialized: now 1786268198188135 us; error 38 us; skew 500 ppm
Aug 09 09:36:38 dist-test-slave-7kzw krb5kdc[13687](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268198, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 09:36:38.191046 14037 init.cc:377] Logged in from keytab as kudu/127.20.125.130@KRBTEST.COM (short username kudu)
I20260809 09:36:38.191941 14037 webserver.cc:460] Webserver started at http://127.20.125.130:44475/ using document root <none> and password file <none>
I20260809 09:36:38.192513 14037 fs_manager.cc:359] Metadata directory not provided
I20260809 09:36:38.192577 14037 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 09:36:38.192834 14037 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 09:36:38.194934 14037 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786267941944493-20982-0/minicluster-data/ts-1/data/instance:
uuid: "3d3876803127410ca87e28ee8fccaba3"
format_stamp: "Formatted at 2026-08-09 09:36:38 on dist-test-slave-7kzw"
server_key: "14d3ea2880bdc6692fcec5b8acf36b7e"
server_key_iv: "a10c79a334be7981c5a986e164b05212"
server_key_version: "encryptionkey@0"
I20260809 09:36:38.195480 14037 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786267941944493-20982-0/minicluster-data/ts-1/wal/instance:
uuid: "3d3876803127410ca87e28ee8fccaba3"
format_stamp: "Formatted at 2026-08-09 09:36:38 on dist-test-slave-7kzw"
server_key: "14d3ea2880bdc6692fcec5b8acf36b7e"
server_key_iv: "a10c79a334be7981c5a986e164b05212"
server_key_version: "encryptionkey@0"
I20260809 09:36:38.199329 14037 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260809 09:36:38.202123 14058 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 09:36:38.203603 14037 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260809 09:36:38.203776 14037 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786267941944493-20982-0/minicluster-data/ts-1/data,/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786267941944493-20982-0/minicluster-data/ts-1/wal
uuid: "3d3876803127410ca87e28ee8fccaba3"
format_stamp: "Formatted at 2026-08-09 09:36:38 on dist-test-slave-7kzw"
server_key: "14d3ea2880bdc6692fcec5b8acf36b7e"
server_key_iv: "a10c79a334be7981c5a986e164b05212"
server_key_version: "encryptionkey@0"
I20260809 09:36:38.203939 14037 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786267941944493-20982-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786267941944493-20982-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786267941944493-20982-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 09:36:38.220479 14037 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 09:36:38.223701 14037 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 09:36:38.223904 14037 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 09:36:38.224542 14037 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 09:36:38.225410 14037 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 09:36:38.225487 14037 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 09:36:38.225564 14037 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 09:36:38.225615 14037 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 09:36:38.235008 14037 rpc_server.cc:307] RPC server started. Bound to: 127.20.125.130:36963
I20260809 09:36:38.235036 14219 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.125.130:36963 every 8 connection(s)
I20260809 09:36:38.235890 14037 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786267941944493-20982-0/minicluster-data/ts-1/data/info.pb
I20260809 09:36:38.243089 20982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu as pid 14037
I20260809 09:36:38.243268 20982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786267941944493-20982-0/minicluster-data/ts-1/wal/instance
I20260809 09:36:38.243583 20982 external_mini_cluster.cc:1428] Setting key 3ef9c002aa97ec4305e4ef9286d94154
Aug 09 09:36:38 dist-test-slave-7kzw krb5kdc[13687](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268198, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.130@KRBTEST.COM for kudu/127.20.125.190@KRBTEST.COM
I20260809 09:36:38.251039 13800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:36:38.237808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.130:60191 (local address 127.20.125.190:42053)
0809 09:36:38.237958 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0809 09:36:38.237962 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 09:36:38.238760 (+ 798us) server_negotiation.cc:407] Connection header received
0809 09:36:38.240187 (+ 1427us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:36:38.240190 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:36:38.240240 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:36:38.240323 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:36:38.241548 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:36:38.242064 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:36:38.243460 (+ 1396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:36:38.243761 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:36:38.243888 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:36:38.246758 (+ 2870us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:36:38.246765 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:36:38.246772 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:36:38.246809 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:36:38.248468 (+ 1659us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:36:38.249157 (+ 689us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:36:38.249160 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:36:38.249161 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:36:38.249248 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:36:38.249658 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:36:38.249660 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:36:38.249661 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:36:38.249866 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:36:38.249950 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0809 09:36:38.250754 (+ 804us) server_negotiation.cc:299] Negotiation successful
0809 09:36:38.250886 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260809 09:36:38.252146 14225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:36:38.238086 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:42053 (local address 127.20.125.130:60191)
0809 09:36:38.238597 (+ 511us) negotiation.cc:107] Waiting for socket to connect
0809 09:36:38.238643 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 09:36:38.239967 (+ 1324us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:36:38.240496 (+ 529us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:36:38.240503 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:36:38.240907 (+ 404us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:36:38.241415 (+ 508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:36:38.241430 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:36:38.242198 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:36:38.242201 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:36:38.243313 (+ 1112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:36:38.243323 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:36:38.243930 (+ 607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:36:38.243935 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:36:38.244151 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:36:38.244917 (+ 766us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 09:36:38.244937 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:36:38.246574 (+ 1637us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 09:36:38.248594 (+ 2020us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:36:38.248601 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:36:38.248616 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:36:38.249040 (+ 424us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:36:38.249348 (+ 308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:36:38.249354 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:36:38.249359 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:36:38.249553 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:36:38.249979 (+ 426us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 09:36:38.250009 (+ 30us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 09:36:38.250356 (+ 347us) client_negotiation.cc:769] Sending connection context
0809 09:36:38.250616 (+ 260us) client_negotiation.cc:240] Negotiation successful
0809 09:36:38.251004 (+ 388us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":103,"threads_started":1}
I20260809 09:36:38.253465 14222 heartbeater.cc:347] Connected to a master server at 127.20.125.190:42053
I20260809 09:36:38.253738 14222 heartbeater.cc:464] Registering TS with master...
I20260809 09:36:38.254258 14222 heartbeater.cc:511] Master 127.20.125.190:42053 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.20.125.131@KRBTEST.COM; defaulting to no policy
I20260809 09:36:38.255475 13750 ts_manager.cc:194] Registered new tserver with Master: 3d3876803127410ca87e28ee8fccaba3 (127.20.125.130:36963)
I20260809 09:36:38.256111 13750 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.125.130@KRBTEST.COM'} at 127.20.125.130:60191
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.125.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.125.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.131.keytab.
Entry for principal kudu/127.20.125.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.131.keytab.
I20260809 09:36:38.273949 20982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
/tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786267941944493-20982-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786267941944493-20982-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786267941944493-20982-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786267941944493-20982-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.125.131:0
--local_ip_for_outbound_sockets=127.20.125.131
--webserver_interface=127.20.125.131
--webserver_port=0
--tserver_master_addrs=127.20.125.190:42053
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.131.keytab
--principal=kudu/127.20.125.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.125.148:35723
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786267941944493-20982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786267941944493-20982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786267941944493-20982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 09:36:38.392333 14228 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 09:36:38.392606 14228 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 09:36:38.392648 14228 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 09:36:38.396624 14228 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 09:36:38.396687 14228 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 09:36:38.396745 14228 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.125.131
I20260809 09:36:38.402180 14228 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.125.148:35723
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786267941944493-20982-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786267941944493-20982-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.131.keytab
--principal=kudu/127.20.125.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.125.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786267941944493-20982-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.125.131
--webserver_port=0
--tserver_master_addrs=127.20.125.190:42053
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14228
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.125.131
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786267941944493-20982-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 08:38:55 UTC on ccycloud.quasar-njjleu.root.comops.site
build id 4575935
I20260809 09:36:38.403250 14228 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 09:36:38.404220 14228 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 09:36:38.414325 14234 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:36:38.414494 14228 server_base.cc:1034] running on GCE node
W20260809 09:36:38.414381 14235 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:36:38.414325 14237 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:36:38.415096 14228 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 09:36:38.415741 14228 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 09:36:38.416937 14228 hybrid_clock.cc:648] HybridClock initialized: now 1786268198416915 us; error 30 us; skew 500 ppm
Aug 09 09:36:38 dist-test-slave-7kzw krb5kdc[13687](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268198, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 09:36:38.420234 14228 init.cc:377] Logged in from keytab as kudu/127.20.125.131@KRBTEST.COM (short username kudu)
I20260809 09:36:38.421120 14228 webserver.cc:460] Webserver started at http://127.20.125.131:42927/ using document root <none> and password file <none>
I20260809 09:36:38.421692 14228 fs_manager.cc:359] Metadata directory not provided
I20260809 09:36:38.421757 14228 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 09:36:38.421931 14228 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 09:36:38.424162 14228 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786267941944493-20982-0/minicluster-data/ts-2/data/instance:
uuid: "68dd2a077d3749c4a56c9dab5cd126d4"
format_stamp: "Formatted at 2026-08-09 09:36:38 on dist-test-slave-7kzw"
server_key: "2ee5f0265287217520302f1fa5a9f64e"
server_key_iv: "180fbcfd543dcad9b9bc154400562136"
server_key_version: "encryptionkey@0"
I20260809 09:36:38.424737 14228 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786267941944493-20982-0/minicluster-data/ts-2/wal/instance:
uuid: "68dd2a077d3749c4a56c9dab5cd126d4"
format_stamp: "Formatted at 2026-08-09 09:36:38 on dist-test-slave-7kzw"
server_key: "2ee5f0265287217520302f1fa5a9f64e"
server_key_iv: "180fbcfd543dcad9b9bc154400562136"
server_key_version: "encryptionkey@0"
I20260809 09:36:38.428773 14228 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.001s
I20260809 09:36:38.431401 14248 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 09:36:38.432689 14228 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 09:36:38.432847 14228 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786267941944493-20982-0/minicluster-data/ts-2/data,/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786267941944493-20982-0/minicluster-data/ts-2/wal
uuid: "68dd2a077d3749c4a56c9dab5cd126d4"
format_stamp: "Formatted at 2026-08-09 09:36:38 on dist-test-slave-7kzw"
server_key: "2ee5f0265287217520302f1fa5a9f64e"
server_key_iv: "180fbcfd543dcad9b9bc154400562136"
server_key_version: "encryptionkey@0"
I20260809 09:36:38.432976 14228 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786267941944493-20982-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786267941944493-20982-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786267941944493-20982-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 09:36:38.467607 14228 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 09:36:38.471406 14228 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 09:36:38.471659 14228 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 09:36:38.472337 14228 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 09:36:38.473258 14228 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 09:36:38.473343 14228 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 09:36:38.473426 14228 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 09:36:38.473485 14228 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 09:36:38.483323 14228 rpc_server.cc:307] RPC server started. Bound to: 127.20.125.131:38635
I20260809 09:36:38.483350 14412 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.125.131:38635 every 8 connection(s)
I20260809 09:36:38.484285 14228 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786267941944493-20982-0/minicluster-data/ts-2/data/info.pb
I20260809 09:36:38.491386 20982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu as pid 14228
I20260809 09:36:38.491545 20982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786267941944493-20982-0/minicluster-data/ts-2/wal/instance
I20260809 09:36:38.491852 20982 external_mini_cluster.cc:1428] Setting key 04cfda0c78ad0b5f0a1a05358f83dc64
Aug 09 09:36:38 dist-test-slave-7kzw krb5kdc[13687](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268198, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.131@KRBTEST.COM for kudu/127.20.125.190@KRBTEST.COM
I20260809 09:36:38.499348 13800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:36:38.486292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.131:36353 (local address 127.20.125.190:42053)
0809 09:36:38.486423 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0809 09:36:38.486427 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 09:36:38.487463 (+ 1036us) server_negotiation.cc:407] Connection header received
0809 09:36:38.488813 (+ 1350us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:36:38.488817 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:36:38.488885 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:36:38.488988 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:36:38.490113 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:36:38.490725 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:36:38.492276 (+ 1551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:36:38.492503 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:36:38.492623 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:36:38.495462 (+ 2839us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:36:38.495465 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:36:38.495467 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:36:38.495495 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:36:38.497227 (+ 1732us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:36:38.497762 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:36:38.497765 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:36:38.497767 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:36:38.497854 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:36:38.498173 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:36:38.498176 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:36:38.498178 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:36:38.498359 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:36:38.498440 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0809 09:36:38.499111 (+ 671us) server_negotiation.cc:299] Negotiation successful
0809 09:36:38.499223 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260809 09:36:38.500346 14419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:36:38.486810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:42053 (local address 127.20.125.131:36353)
0809 09:36:38.487314 (+ 504us) negotiation.cc:107] Waiting for socket to connect
0809 09:36:38.487356 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 09:36:38.488575 (+ 1219us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:36:38.489157 (+ 582us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:36:38.489164 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:36:38.489509 (+ 345us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:36:38.489963 (+ 454us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:36:38.489976 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:36:38.490858 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:36:38.490861 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:36:38.492124 (+ 1263us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:36:38.492135 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:36:38.492711 (+ 576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:36:38.492716 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:36:38.492911 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:36:38.493528 (+ 617us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 09:36:38.493545 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:36:38.495233 (+ 1688us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 09:36:38.497354 (+ 2121us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:36:38.497359 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:36:38.497371 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:36:38.497632 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:36:38.497947 (+ 315us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:36:38.497949 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:36:38.497950 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:36:38.498086 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:36:38.498589 (+ 503us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 09:36:38.498596 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 09:36:38.498863 (+ 267us) client_negotiation.cc:769] Sending connection context
0809 09:36:38.499050 (+ 187us) client_negotiation.cc:240] Negotiation successful
0809 09:36:38.499352 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":288,"thread_start_us":121,"threads_started":1}
I20260809 09:36:38.501652 14415 heartbeater.cc:347] Connected to a master server at 127.20.125.190:42053
I20260809 09:36:38.501904 14415 heartbeater.cc:464] Registering TS with master...
I20260809 09:36:38.502377 14415 heartbeater.cc:511] Master 127.20.125.190:42053 requested a full tablet report, sending...
I20260809 09:36:38.503531 13750 ts_manager.cc:194] Registered new tserver with Master: 68dd2a077d3749c4a56c9dab5cd126d4 (127.20.125.131:38635)
I20260809 09:36:38.504096 13750 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.125.131@KRBTEST.COM'} at 127.20.125.131:36353
I20260809 09:36:38.506011 20982 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 09:36:38.515727 13800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:36:38.508368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36680 (local address 127.20.125.190:42053)
0809 09:36:38.508492 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0809 09:36:38.508497 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 09:36:38.508679 (+ 182us) server_negotiation.cc:407] Connection header received
0809 09:36:38.508766 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:36:38.508768 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:36:38.508806 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:36:38.508887 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:36:38.509653 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:36:38.510140 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:36:38.510963 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:36:38.511165 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:36:38.511263 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:36:38.512573 (+ 1310us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:36:38.512575 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:36:38.512577 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:36:38.512599 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:36:38.514049 (+ 1450us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:36:38.514606 (+ 557us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:36:38.514611 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:36:38.514612 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:36:38.514701 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:36:38.515002 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:36:38.515005 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:36:38.515006 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:36:38.515238 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:36:38.515351 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0809 09:36:38.515524 (+ 173us) server_negotiation.cc:299] Negotiation successful
0809 09:36:38.515613 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260809 09:36:38.518350 13750 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36680:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 09:36:38.521306 13750 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 09:36:38.535923 14433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:36:38.529946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54516 (local address 127.20.125.131:38635)
0809 09:36:38.531105 (+ 1159us) server_negotiation.cc:206] Beginning negotiation
0809 09:36:38.531120 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 09:36:38.531146 (+ 26us) server_negotiation.cc:407] Connection header received
0809 09:36:38.531237 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:36:38.531242 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:36:38.531449 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:36:38.531948 (+ 499us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 09:36:38.532095 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:36:38.532979 (+ 884us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:36:38.534454 (+ 1475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:36:38.535254 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:36:38.535392 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:36:38.535528 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0809 09:36:38.535686 (+ 158us) server_negotiation.cc:299] Negotiation successful
0809 09:36:38.535780 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1062,"thread_start_us":119,"threads_started":1}
I20260809 09:36:38.535913 14432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:36:38.529855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.131:38635 (local address 127.0.0.1:54516)
0809 09:36:38.530490 (+ 635us) negotiation.cc:107] Waiting for socket to connect
0809 09:36:38.530541 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0809 09:36:38.530662 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:36:38.531626 (+ 964us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:36:38.531632 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:36:38.531655 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 09:36:38.531800 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:36:38.531807 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:36:38.533111 (+ 1304us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:36:38.533117 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:36:38.534303 (+ 1186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:36:38.534313 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:36:38.535377 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:36:38.535381 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:36:38.535497 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:36:38.535536 (+ 39us) client_negotiation.cc:769] Sending connection context
0809 09:36:38.535639 (+ 103us) client_negotiation.cc:240] Negotiation successful
0809 09:36:38.535760 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":449,"thread_start_us":87,"threads_started":1}
I20260809 09:36:38.536262 14434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:36:38.530203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32876 (local address 127.20.125.129:37127)
0809 09:36:38.530878 (+ 675us) server_negotiation.cc:206] Beginning negotiation
0809 09:36:38.530887 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 09:36:38.531172 (+ 285us) server_negotiation.cc:407] Connection header received
0809 09:36:38.531377 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:36:38.531382 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:36:38.531540 (+ 158us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:36:38.531655 (+ 115us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 09:36:38.532079 (+ 424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:36:38.533249 (+ 1170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:36:38.534676 (+ 1427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:36:38.535407 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:36:38.535526 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:36:38.535646 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0809 09:36:38.535973 (+ 327us) server_negotiation.cc:299] Negotiation successful
0809 09:36:38.536076 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":379,"thread_start_us":124,"threads_started":1}
I20260809 09:36:38.536526 14436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:36:38.530639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.130:36963 (local address 127.0.0.1:55538)
0809 09:36:38.531431 (+ 792us) negotiation.cc:107] Waiting for socket to connect
0809 09:36:38.531445 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 09:36:38.531526 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:36:38.532251 (+ 725us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:36:38.532254 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:36:38.532274 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 09:36:38.532369 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:36:38.532375 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:36:38.533647 (+ 1272us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:36:38.533651 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:36:38.534868 (+ 1217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:36:38.534881 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:36:38.536102 (+ 1221us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:36:38.536106 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:36:38.536207 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:36:38.536240 (+ 33us) client_negotiation.cc:769] Sending connection context
0809 09:36:38.536318 (+ 78us) client_negotiation.cc:240] Negotiation successful
0809 09:36:38.536392 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":733,"mutex_wait_us":88,"thread_start_us":100,"threads_started":1}
I20260809 09:36:38.536580 14435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:36:38.530541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.129:37127 (local address 127.0.0.1:32876)
0809 09:36:38.531071 (+ 530us) negotiation.cc:107] Waiting for socket to connect
0809 09:36:38.531086 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 09:36:38.531242 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:36:38.531836 (+ 594us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:36:38.531841 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:36:38.531861 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 09:36:38.531953 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:36:38.531958 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:36:38.533381 (+ 1423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:36:38.533385 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:36:38.534572 (+ 1187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:36:38.534585 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:36:38.535714 (+ 1129us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:36:38.535718 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:36:38.535807 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:36:38.535837 (+ 30us) client_negotiation.cc:769] Sending connection context
0809 09:36:38.536412 (+ 575us) client_negotiation.cc:240] Negotiation successful
0809 09:36:38.536471 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":461,"mutex_wait_us":2,"thread_start_us":100,"threads_started":1}
I20260809 09:36:38.537058 14437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:36:38.531008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55538 (local address 127.20.125.130:36963)
0809 09:36:38.531790 (+ 782us) server_negotiation.cc:206] Beginning negotiation
0809 09:36:38.531802 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 09:36:38.531825 (+ 23us) server_negotiation.cc:407] Connection header received
0809 09:36:38.531897 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:36:38.531900 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:36:38.532087 (+ 187us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:36:38.532248 (+ 161us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 09:36:38.532465 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:36:38.533512 (+ 1047us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:36:38.535002 (+ 1490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:36:38.535802 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:36:38.535930 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:36:38.536041 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0809 09:36:38.536766 (+ 725us) server_negotiation.cc:299] Negotiation successful
0809 09:36:38.536880 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":667,"thread_start_us":120,"threads_started":1}
I20260809 09:36:38.539525 14125 tablet_service.cc:1467] Processing CreateTablet for tablet 1b67d32d85294ca28dba37d10ba37694 (DEFAULT_TABLE table=test-table [id=39705ed024834a1fab5dcfd6a62ee9ed]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 09:36:38.539530 14321 tablet_service.cc:1467] Processing CreateTablet for tablet 1b67d32d85294ca28dba37d10ba37694 (DEFAULT_TABLE table=test-table [id=39705ed024834a1fab5dcfd6a62ee9ed]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 09:36:38.539527 13934 tablet_service.cc:1467] Processing CreateTablet for tablet 1b67d32d85294ca28dba37d10ba37694 (DEFAULT_TABLE table=test-table [id=39705ed024834a1fab5dcfd6a62ee9ed]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 09:36:38.540514 14125 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1b67d32d85294ca28dba37d10ba37694. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 09:36:38.540541 13934 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1b67d32d85294ca28dba37d10ba37694. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 09:36:38.540541 14321 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1b67d32d85294ca28dba37d10ba37694. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 09:36:38.547029 14438 tablet_bootstrap.cc:492] T 1b67d32d85294ca28dba37d10ba37694 P 3d3876803127410ca87e28ee8fccaba3: Bootstrap starting.
I20260809 09:36:38.547962 14439 tablet_bootstrap.cc:492] T 1b67d32d85294ca28dba37d10ba37694 P 74cb73ba683f49a18f0b3af5f5ed9ebd: Bootstrap starting.
I20260809 09:36:38.548345 14440 tablet_bootstrap.cc:492] T 1b67d32d85294ca28dba37d10ba37694 P 68dd2a077d3749c4a56c9dab5cd126d4: Bootstrap starting.
I20260809 09:36:38.549438 14438 tablet_bootstrap.cc:654] T 1b67d32d85294ca28dba37d10ba37694 P 3d3876803127410ca87e28ee8fccaba3: Neither blocks nor log segments found. Creating new log.
I20260809 09:36:38.550004 14440 tablet_bootstrap.cc:654] T 1b67d32d85294ca28dba37d10ba37694 P 68dd2a077d3749c4a56c9dab5cd126d4: Neither blocks nor log segments found. Creating new log.
I20260809 09:36:38.550047 14439 tablet_bootstrap.cc:654] T 1b67d32d85294ca28dba37d10ba37694 P 74cb73ba683f49a18f0b3af5f5ed9ebd: Neither blocks nor log segments found. Creating new log.
I20260809 09:36:38.550856 14438 log.cc:824] T 1b67d32d85294ca28dba37d10ba37694 P 3d3876803127410ca87e28ee8fccaba3: Log is configured to *not* fsync() on all Append() calls
I20260809 09:36:38.550856 14439 log.cc:824] T 1b67d32d85294ca28dba37d10ba37694 P 74cb73ba683f49a18f0b3af5f5ed9ebd: Log is configured to *not* fsync() on all Append() calls
I20260809 09:36:38.551266 14440 log.cc:824] T 1b67d32d85294ca28dba37d10ba37694 P 68dd2a077d3749c4a56c9dab5cd126d4: Log is configured to *not* fsync() on all Append() calls
I20260809 09:36:38.553086 14439 tablet_bootstrap.cc:492] T 1b67d32d85294ca28dba37d10ba37694 P 74cb73ba683f49a18f0b3af5f5ed9ebd: No bootstrap required, opened a new log
I20260809 09:36:38.553086 14438 tablet_bootstrap.cc:492] T 1b67d32d85294ca28dba37d10ba37694 P 3d3876803127410ca87e28ee8fccaba3: No bootstrap required, opened a new log
I20260809 09:36:38.553238 14440 tablet_bootstrap.cc:492] T 1b67d32d85294ca28dba37d10ba37694 P 68dd2a077d3749c4a56c9dab5cd126d4: No bootstrap required, opened a new log
I20260809 09:36:38.553282 14439 ts_tablet_manager.cc:1397] T 1b67d32d85294ca28dba37d10ba37694 P 74cb73ba683f49a18f0b3af5f5ed9ebd: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260809 09:36:38.553282 14438 ts_tablet_manager.cc:1397] T 1b67d32d85294ca28dba37d10ba37694 P 3d3876803127410ca87e28ee8fccaba3: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260809 09:36:38.553445 14440 ts_tablet_manager.cc:1397] T 1b67d32d85294ca28dba37d10ba37694 P 68dd2a077d3749c4a56c9dab5cd126d4: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260809 09:36:38.556238 14440 raft_consensus.cc:348] T 1b67d32d85294ca28dba37d10ba37694 P 68dd2a077d3749c4a56c9dab5cd126d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68dd2a077d3749c4a56c9dab5cd126d4" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 38635 } } peers { permanent_uuid: "74cb73ba683f49a18f0b3af5f5ed9ebd" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 37127 } } peers { permanent_uuid: "3d3876803127410ca87e28ee8fccaba3" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 36963 } }
I20260809 09:36:38.556474 14440 raft_consensus.cc:374] T 1b67d32d85294ca28dba37d10ba37694 P 68dd2a077d3749c4a56c9dab5cd126d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 09:36:38.556558 14440 raft_consensus.cc:729] T 1b67d32d85294ca28dba37d10ba37694 P 68dd2a077d3749c4a56c9dab5cd126d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68dd2a077d3749c4a56c9dab5cd126d4, State: Initialized, Role: FOLLOWER
I20260809 09:36:38.557029 14440 consensus_queue.cc:260] T 1b67d32d85294ca28dba37d10ba37694 P 68dd2a077d3749c4a56c9dab5cd126d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68dd2a077d3749c4a56c9dab5cd126d4" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 38635 } } peers { permanent_uuid: "74cb73ba683f49a18f0b3af5f5ed9ebd" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 37127 } } peers { permanent_uuid: "3d3876803127410ca87e28ee8fccaba3" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 36963 } }
I20260809 09:36:38.557191 14438 raft_consensus.cc:348] T 1b67d32d85294ca28dba37d10ba37694 P 3d3876803127410ca87e28ee8fccaba3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68dd2a077d3749c4a56c9dab5cd126d4" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 38635 } } peers { permanent_uuid: "74cb73ba683f49a18f0b3af5f5ed9ebd" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 37127 } } peers { permanent_uuid: "3d3876803127410ca87e28ee8fccaba3" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 36963 } }
I20260809 09:36:38.557497 14438 raft_consensus.cc:374] T 1b67d32d85294ca28dba37d10ba37694 P 3d3876803127410ca87e28ee8fccaba3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 09:36:38.557610 14438 raft_consensus.cc:729] T 1b67d32d85294ca28dba37d10ba37694 P 3d3876803127410ca87e28ee8fccaba3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d3876803127410ca87e28ee8fccaba3, State: Initialized, Role: FOLLOWER
I20260809 09:36:38.557713 14415 heartbeater.cc:503] Master 127.20.125.190:42053 was elected leader, sending a full tablet report...
I20260809 09:36:38.558051 14440 ts_tablet_manager.cc:1428] T 1b67d32d85294ca28dba37d10ba37694 P 68dd2a077d3749c4a56c9dab5cd126d4: Time spent starting tablet: real 0.005s user 0.001s sys 0.004s
I20260809 09:36:38.558043 14438 consensus_queue.cc:260] T 1b67d32d85294ca28dba37d10ba37694 P 3d3876803127410ca87e28ee8fccaba3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68dd2a077d3749c4a56c9dab5cd126d4" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 38635 } } peers { permanent_uuid: "74cb73ba683f49a18f0b3af5f5ed9ebd" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 37127 } } peers { permanent_uuid: "3d3876803127410ca87e28ee8fccaba3" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 36963 } }
I20260809 09:36:38.558743 14222 heartbeater.cc:503] Master 127.20.125.190:42053 was elected leader, sending a full tablet report...
I20260809 09:36:38.559998 14438 ts_tablet_manager.cc:1428] T 1b67d32d85294ca28dba37d10ba37694 P 3d3876803127410ca87e28ee8fccaba3: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260809 09:36:38.557525 14439 raft_consensus.cc:348] T 1b67d32d85294ca28dba37d10ba37694 P 74cb73ba683f49a18f0b3af5f5ed9ebd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68dd2a077d3749c4a56c9dab5cd126d4" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 38635 } } peers { permanent_uuid: "74cb73ba683f49a18f0b3af5f5ed9ebd" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 37127 } } peers { permanent_uuid: "3d3876803127410ca87e28ee8fccaba3" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 36963 } }
I20260809 09:36:38.560490 14439 raft_consensus.cc:374] T 1b67d32d85294ca28dba37d10ba37694 P 74cb73ba683f49a18f0b3af5f5ed9ebd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 09:36:38.560557 14439 raft_consensus.cc:729] T 1b67d32d85294ca28dba37d10ba37694 P 74cb73ba683f49a18f0b3af5f5ed9ebd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74cb73ba683f49a18f0b3af5f5ed9ebd, State: Initialized, Role: FOLLOWER
I20260809 09:36:38.560995 14439 consensus_queue.cc:260] T 1b67d32d85294ca28dba37d10ba37694 P 74cb73ba683f49a18f0b3af5f5ed9ebd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68dd2a077d3749c4a56c9dab5cd126d4" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 38635 } } peers { permanent_uuid: "74cb73ba683f49a18f0b3af5f5ed9ebd" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 37127 } } peers { permanent_uuid: "3d3876803127410ca87e28ee8fccaba3" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 36963 } }
I20260809 09:36:38.561672 14030 heartbeater.cc:503] Master 127.20.125.190:42053 was elected leader, sending a full tablet report...
I20260809 09:36:38.561968 14439 ts_tablet_manager.cc:1428] T 1b67d32d85294ca28dba37d10ba37694 P 74cb73ba683f49a18f0b3af5f5ed9ebd: Time spent starting tablet: real 0.009s user 0.007s sys 0.000s
I20260809 09:36:38.575323 14447 raft_consensus.cc:482] T 1b67d32d85294ca28dba37d10ba37694 P 3d3876803127410ca87e28ee8fccaba3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 09:36:38.575464 14447 raft_consensus.cc:504] T 1b67d32d85294ca28dba37d10ba37694 P 3d3876803127410ca87e28ee8fccaba3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68dd2a077d3749c4a56c9dab5cd126d4" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 38635 } } peers { permanent_uuid: "74cb73ba683f49a18f0b3af5f5ed9ebd" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 37127 } } peers { permanent_uuid: "3d3876803127410ca87e28ee8fccaba3" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 36963 } }
I20260809 09:36:38.576433 14447 leader_election.cc:288] T 1b67d32d85294ca28dba37d10ba37694 P 3d3876803127410ca87e28ee8fccaba3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 68dd2a077d3749c4a56c9dab5cd126d4 (127.20.125.131:38635), 74cb73ba683f49a18f0b3af5f5ed9ebd (127.20.125.129:37127)
I20260809 09:36:38.580318 14434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:36:38.576790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.130:58839 (local address 127.20.125.129:37127)
0809 09:36:38.576906 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0809 09:36:38.576912 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 09:36:38.577069 (+ 157us) server_negotiation.cc:407] Connection header received
0809 09:36:38.577204 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:36:38.577209 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:36:38.577259 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:36:38.577330 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 09:36:38.577657 (+ 327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:36:38.578219 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:36:38.579298 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:36:38.579707 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:36:38.579836 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:36:38.579890 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0809 09:36:38.580118 (+ 228us) server_negotiation.cc:299] Negotiation successful
0809 09:36:38.580197 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260809 09:36:38.580318 14433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:36:38.576703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.130:55817 (local address 127.20.125.131:38635)
0809 09:36:38.576824 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0809 09:36:38.576829 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 09:36:38.576946 (+ 117us) server_negotiation.cc:407] Connection header received
0809 09:36:38.577072 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:36:38.577075 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:36:38.577127 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:36:38.577210 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 09:36:38.577480 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:36:38.578160 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:36:38.579295 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:36:38.579695 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:36:38.579824 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:36:38.579880 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0809 09:36:38.580113 (+ 233us) server_negotiation.cc:299] Negotiation successful
0809 09:36:38.580190 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260809 09:36:38.580320 14225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:36:38.576623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.131:38635 (local address 127.20.125.130:55817)
0809 09:36:38.576832 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0809 09:36:38.576849 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 09:36:38.576943 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:36:38.577232 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:36:38.577235 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:36:38.577255 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 09:36:38.577375 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:36:38.577383 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:36:38.578279 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:36:38.578283 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:36:38.579162 (+ 879us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:36:38.579178 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:36:38.579824 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:36:38.579828 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:36:38.579974 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:36:38.580007 (+ 33us) client_negotiation.cc:769] Sending connection context
0809 09:36:38.580097 (+ 90us) client_negotiation.cc:240] Negotiation successful
0809 09:36:38.580205 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"mutex_wait_us":43}
I20260809 09:36:38.580334 14453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:36:38.576691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.129:37127 (local address 127.20.125.130:58839)
0809 09:36:38.576969 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0809 09:36:38.576983 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 09:36:38.577093 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:36:38.577372 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:36:38.577375 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:36:38.577392 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 09:36:38.577527 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:36:38.577534 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:36:38.578337 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:36:38.578342 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:36:38.579168 (+ 826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:36:38.579177 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:36:38.579831 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:36:38.579835 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:36:38.579982 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:36:38.580011 (+ 29us) client_negotiation.cc:769] Sending connection context
0809 09:36:38.580100 (+ 89us) client_negotiation.cc:240] Negotiation successful
0809 09:36:38.580191 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":225,"mutex_wait_us":28,"thread_start_us":110,"threads_started":1}
I20260809 09:36:38.580933 13959 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1b67d32d85294ca28dba37d10ba37694" candidate_uuid: "3d3876803127410ca87e28ee8fccaba3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74cb73ba683f49a18f0b3af5f5ed9ebd" is_pre_election: true
I20260809 09:36:38.581097 14352 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1b67d32d85294ca28dba37d10ba37694" candidate_uuid: "3d3876803127410ca87e28ee8fccaba3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68dd2a077d3749c4a56c9dab5cd126d4" is_pre_election: true
I20260809 09:36:38.581218 13959 raft_consensus.cc:2454] T 1b67d32d85294ca28dba37d10ba37694 P 74cb73ba683f49a18f0b3af5f5ed9ebd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d3876803127410ca87e28ee8fccaba3 in term 0.
I20260809 09:36:38.581393 14352 raft_consensus.cc:2454] T 1b67d32d85294ca28dba37d10ba37694 P 68dd2a077d3749c4a56c9dab5cd126d4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d3876803127410ca87e28ee8fccaba3 in term 0.
I20260809 09:36:38.581736 14059 leader_election.cc:302] T 1b67d32d85294ca28dba37d10ba37694 P 3d3876803127410ca87e28ee8fccaba3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d3876803127410ca87e28ee8fccaba3, 74cb73ba683f49a18f0b3af5f5ed9ebd; no voters:
I20260809 09:36:38.581997 14447 raft_consensus.cc:2781] T 1b67d32d85294ca28dba37d10ba37694 P 3d3876803127410ca87e28ee8fccaba3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 09:36:38.582079 14447 raft_consensus.cc:482] T 1b67d32d85294ca28dba37d10ba37694 P 3d3876803127410ca87e28ee8fccaba3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 09:36:38.582130 14447 raft_consensus.cc:3037] T 1b67d32d85294ca28dba37d10ba37694 P 3d3876803127410ca87e28ee8fccaba3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 09:36:38.583216 14447 raft_consensus.cc:504] T 1b67d32d85294ca28dba37d10ba37694 P 3d3876803127410ca87e28ee8fccaba3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68dd2a077d3749c4a56c9dab5cd126d4" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 38635 } } peers { permanent_uuid: "74cb73ba683f49a18f0b3af5f5ed9ebd" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 37127 } } peers { permanent_uuid: "3d3876803127410ca87e28ee8fccaba3" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 36963 } }
I20260809 09:36:38.583529 14447 leader_election.cc:288] T 1b67d32d85294ca28dba37d10ba37694 P 3d3876803127410ca87e28ee8fccaba3 [CANDIDATE]: Term 1 election: Requested vote from peers 68dd2a077d3749c4a56c9dab5cd126d4 (127.20.125.131:38635), 74cb73ba683f49a18f0b3af5f5ed9ebd (127.20.125.129:37127)
I20260809 09:36:38.583797 14352 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1b67d32d85294ca28dba37d10ba37694" candidate_uuid: "3d3876803127410ca87e28ee8fccaba3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68dd2a077d3749c4a56c9dab5cd126d4"
I20260809 09:36:38.583927 14352 raft_consensus.cc:3037] T 1b67d32d85294ca28dba37d10ba37694 P 68dd2a077d3749c4a56c9dab5cd126d4 [term 0 FOLLOWER]: Advancing to term 1
I20260809 09:36:38.583878 13959 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1b67d32d85294ca28dba37d10ba37694" candidate_uuid: "3d3876803127410ca87e28ee8fccaba3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74cb73ba683f49a18f0b3af5f5ed9ebd"
I20260809 09:36:38.584053 13959 raft_consensus.cc:3037] T 1b67d32d85294ca28dba37d10ba37694 P 74cb73ba683f49a18f0b3af5f5ed9ebd [term 0 FOLLOWER]: Advancing to term 1
I20260809 09:36:38.585144 14352 raft_consensus.cc:2454] T 1b67d32d85294ca28dba37d10ba37694 P 68dd2a077d3749c4a56c9dab5cd126d4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d3876803127410ca87e28ee8fccaba3 in term 1.
I20260809 09:36:38.585216 13959 raft_consensus.cc:2454] T 1b67d32d85294ca28dba37d10ba37694 P 74cb73ba683f49a18f0b3af5f5ed9ebd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d3876803127410ca87e28ee8fccaba3 in term 1.
I20260809 09:36:38.585469 14061 leader_election.cc:302] T 1b67d32d85294ca28dba37d10ba37694 P 3d3876803127410ca87e28ee8fccaba3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d3876803127410ca87e28ee8fccaba3, 68dd2a077d3749c4a56c9dab5cd126d4; no voters:
I20260809 09:36:38.585657 14447 raft_consensus.cc:2781] T 1b67d32d85294ca28dba37d10ba37694 P 3d3876803127410ca87e28ee8fccaba3 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 09:36:38.585917 14447 raft_consensus.cc:686] T 1b67d32d85294ca28dba37d10ba37694 P 3d3876803127410ca87e28ee8fccaba3 [term 1 LEADER]: Becoming Leader. State: Replica: 3d3876803127410ca87e28ee8fccaba3, State: Running, Role: LEADER
I20260809 09:36:38.586259 14447 consensus_queue.cc:237] T 1b67d32d85294ca28dba37d10ba37694 P 3d3876803127410ca87e28ee8fccaba3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68dd2a077d3749c4a56c9dab5cd126d4" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 38635 } } peers { permanent_uuid: "74cb73ba683f49a18f0b3af5f5ed9ebd" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 37127 } } peers { permanent_uuid: "3d3876803127410ca87e28ee8fccaba3" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 36963 } }
I20260809 09:36:38.589252 13749 catalog_manager.cc:5495] T 1b67d32d85294ca28dba37d10ba37694 P 3d3876803127410ca87e28ee8fccaba3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3d3876803127410ca87e28ee8fccaba3 (127.20.125.130). New cstate: current_term: 1 leader_uuid: "3d3876803127410ca87e28ee8fccaba3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68dd2a077d3749c4a56c9dab5cd126d4" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 38635 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "74cb73ba683f49a18f0b3af5f5ed9ebd" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 37127 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d3876803127410ca87e28ee8fccaba3" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 36963 } health_report { overall_health: HEALTHY } } }
I20260809 09:36:38.614869 14437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:36:38.611196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55542 (local address 127.20.125.130:36963)
0809 09:36:38.611335 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0809 09:36:38.611342 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 09:36:38.611357 (+ 15us) server_negotiation.cc:407] Connection header received
0809 09:36:38.611545 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:36:38.611548 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:36:38.611602 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:36:38.611757 (+ 155us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 09:36:38.612004 (+ 247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:36:38.612504 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:36:38.613366 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:36:38.613585 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:36:38.613708 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:36:38.614014 (+ 306us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 09:36:38.614408 (+ 394us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 09:36:38.614584 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0809 09:36:38.614656 (+ 72us) server_negotiation.cc:299] Negotiation successful
0809 09:36:38.614718 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260809 09:36:38.620615 13959 raft_consensus.cc:1261] T 1b67d32d85294ca28dba37d10ba37694 P 74cb73ba683f49a18f0b3af5f5ed9ebd [term 1 FOLLOWER]: Refusing update from remote peer 3d3876803127410ca87e28ee8fccaba3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 09:36:38.620615 14352 raft_consensus.cc:1261] T 1b67d32d85294ca28dba37d10ba37694 P 68dd2a077d3749c4a56c9dab5cd126d4 [term 1 FOLLOWER]: Refusing update from remote peer 3d3876803127410ca87e28ee8fccaba3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 09:36:38.621320 14447 consensus_queue.cc:1035] T 1b67d32d85294ca28dba37d10ba37694 P 3d3876803127410ca87e28ee8fccaba3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74cb73ba683f49a18f0b3af5f5ed9ebd" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 37127 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 09:36:38.621512 14455 consensus_queue.cc:1035] T 1b67d32d85294ca28dba37d10ba37694 P 3d3876803127410ca87e28ee8fccaba3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "68dd2a077d3749c4a56c9dab5cd126d4" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 38635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 09:36:38.629215 14459 mvcc.cc:205] Tried to move back new op lower bound from 7316554541542920192 to 7316554541412020224. Current Snapshot: MvccSnapshot[applied={T|T < 7316554541542920192}]
I20260809 09:36:38.639816 13748 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36680:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 09:36:38.640021 13748 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36680:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 09:36:38.642648 13748 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 40ae5ae7dd2e4ef8b86807697ff48b80: Sending DeleteTablet for 3 replicas of tablet 1b67d32d85294ca28dba37d10ba37694
I20260809 09:36:38.643415 13934 tablet_service.cc:1514] Processing DeleteTablet for tablet 1b67d32d85294ca28dba37d10ba37694 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 09:36:38 UTC) from {username='kudu', principal='kudu/127.20.125.190@KRBTEST.COM'} at 127.0.0.1:32876
I20260809 09:36:38.643445 14321 tablet_service.cc:1514] Processing DeleteTablet for tablet 1b67d32d85294ca28dba37d10ba37694 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 09:36:38 UTC) from {username='kudu', principal='kudu/127.20.125.190@KRBTEST.COM'} at 127.0.0.1:54516
I20260809 09:36:38.643604 14125 tablet_service.cc:1514] Processing DeleteTablet for tablet 1b67d32d85294ca28dba37d10ba37694 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 09:36:38 UTC) from {username='kudu', principal='kudu/127.20.125.190@KRBTEST.COM'} at 127.0.0.1:55538
I20260809 09:36:38.643949 14472 tablet_replica.cc:330] T 1b67d32d85294ca28dba37d10ba37694 P 68dd2a077d3749c4a56c9dab5cd126d4: stopping tablet replica
I20260809 09:36:38.644271 14472 raft_consensus.cc:2229] T 1b67d32d85294ca28dba37d10ba37694 P 68dd2a077d3749c4a56c9dab5cd126d4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 09:36:38.644318 14476 tablet_replica.cc:330] T 1b67d32d85294ca28dba37d10ba37694 P 74cb73ba683f49a18f0b3af5f5ed9ebd: stopping tablet replica
I20260809 09:36:38.644585 20982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu with pid 13843
I20260809 09:36:38.644658 14472 raft_consensus.cc:2258] T 1b67d32d85294ca28dba37d10ba37694 P 68dd2a077d3749c4a56c9dab5cd126d4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 09:36:38.644706 14476 raft_consensus.cc:2229] T 1b67d32d85294ca28dba37d10ba37694 P 74cb73ba683f49a18f0b3af5f5ed9ebd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 09:36:38.644984 14476 raft_consensus.cc:2258] T 1b67d32d85294ca28dba37d10ba37694 P 74cb73ba683f49a18f0b3af5f5ed9ebd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 09:36:38.645314 14477 tablet_replica.cc:330] T 1b67d32d85294ca28dba37d10ba37694 P 3d3876803127410ca87e28ee8fccaba3: stopping tablet replica
I20260809 09:36:38.645618 14477 raft_consensus.cc:2229] T 1b67d32d85294ca28dba37d10ba37694 P 3d3876803127410ca87e28ee8fccaba3 [term 1 LEADER]: Raft consensus shutting down.
I20260809 09:36:38.646080 14477 raft_consensus.cc:2258] T 1b67d32d85294ca28dba37d10ba37694 P 3d3876803127410ca87e28ee8fccaba3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 09:36:38.652143 14472 ts_tablet_manager.cc:1905] T 1b67d32d85294ca28dba37d10ba37694 P 68dd2a077d3749c4a56c9dab5cd126d4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 09:36:38.655171 14472 ts_tablet_manager.cc:1918] T 1b67d32d85294ca28dba37d10ba37694 P 68dd2a077d3749c4a56c9dab5cd126d4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 09:36:38.655280 14472 log.cc:1196] T 1b67d32d85294ca28dba37d10ba37694 P 68dd2a077d3749c4a56c9dab5cd126d4: Deleting WAL directory at /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786267941944493-20982-0/minicluster-data/ts-2/wal/wals/1b67d32d85294ca28dba37d10ba37694
I20260809 09:36:38.655690 14472 ts_tablet_manager.cc:1939] T 1b67d32d85294ca28dba37d10ba37694 P 68dd2a077d3749c4a56c9dab5cd126d4: Deleting consensus metadata
I20260809 09:36:38.657019 13727 catalog_manager.cc:4841] TS 68dd2a077d3749c4a56c9dab5cd126d4 (127.20.125.131:38635): tablet 1b67d32d85294ca28dba37d10ba37694 (table test-table [id=39705ed024834a1fab5dcfd6a62ee9ed]) successfully deleted
W20260809 09:36:38.657393 13724 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.20.125.129:37127 (error 108)
I20260809 09:36:38.658308 20982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu with pid 14037
I20260809 09:36:38.658531 14436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:36:38.658171 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.129:37127 (local address 127.0.0.1:32884)
0809 09:36:38.658347 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0809 09:36:38.658416 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.129:37127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97}
W20260809 09:36:38.661368 13724 catalog_manager.cc:4584] TS 74cb73ba683f49a18f0b3af5f5ed9ebd (127.20.125.129:37127): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 1b67d32d85294ca28dba37d10ba37694: Network error: Client connection negotiation failed: client connection to 127.20.125.129:37127: connect: Connection refused (error 111)
I20260809 09:36:38.669602 20982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu with pid 14228
I20260809 09:36:38.669999 14436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:36:38.669713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.130:36963 (local address 127.0.0.1:55558)
0809 09:36:38.669829 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0809 09:36:38.669896 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.130:36963: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260809 09:36:38.690442 20982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu with pid 13705
2026-08-09T09:36:38Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2523 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-09T09:36:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T09:36:38Z Disabled control of system clock
I20260809 09:36:38.725467 20982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
/tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786267941944493-20982-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786267941944493-20982-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786267941944493-20982-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786267941944493-20982-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.125.190:45191
--webserver_interface=127.20.125.190
--webserver_port=0
--builtin_ntp_servers=127.20.125.148:38147
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.125.190:45191
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 09:36:38.844092 14486 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 09:36:38.844373 14486 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 09:36:38.844409 14486 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 09:36:38.848490 14486 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 09:36:38.848551 14486 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 09:36:38.848567 14486 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 09:36:38.848590 14486 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 09:36:38.848606 14486 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 09:36:38.853473 14486 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.125.148:38147
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786267941944493-20982-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786267941944493-20982-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.125.190:45191
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.125.190:45191
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786267941944493-20982-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.125.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14486
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786267941944493-20982-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 08:38:55 UTC on ccycloud.quasar-njjleu.root.comops.site
build id 4575935
I20260809 09:36:38.854637 14486 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 09:36:38.856047 14486 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 09:36:38.864444 14494 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:36:38.864451 14493 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:36:38.864463 14496 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:36:38.864845 14486 server_base.cc:1034] running on GCE node
I20260809 09:36:38.865476 14486 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 09:36:38.866641 14486 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 09:36:38.867832 14486 hybrid_clock.cc:648] HybridClock initialized: now 1786268198867805 us; error 41 us; skew 500 ppm
I20260809 09:36:38.869740 14486 webserver.cc:460] Webserver started at http://127.20.125.190:43593/ using document root <none> and password file <none>
I20260809 09:36:38.870314 14486 fs_manager.cc:359] Metadata directory not provided
I20260809 09:36:38.870412 14486 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 09:36:38.870673 14486 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 09:36:38.872658 14486 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786267941944493-20982-0/minicluster-data/master-0/data/instance:
uuid: "a761fd3aa90e40c68d587c1e199afdab"
format_stamp: "Formatted at 2026-08-09 09:36:38 on dist-test-slave-7kzw"
server_key: "6b9e3eabc0e8fdca7696ad624e8a188f"
server_key_iv: "ab9f0de3177e92a9261b233ec55c140a"
server_key_version: "encryptionkey@0"
I20260809 09:36:38.873239 14486 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786267941944493-20982-0/minicluster-data/master-0/wal/instance:
uuid: "a761fd3aa90e40c68d587c1e199afdab"
format_stamp: "Formatted at 2026-08-09 09:36:38 on dist-test-slave-7kzw"
server_key: "6b9e3eabc0e8fdca7696ad624e8a188f"
server_key_iv: "ab9f0de3177e92a9261b233ec55c140a"
server_key_version: "encryptionkey@0"
I20260809 09:36:38.876901 14486 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260809 09:36:38.879419 14507 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 09:36:38.880646 14486 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 09:36:38.880820 14486 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786267941944493-20982-0/minicluster-data/master-0/data,/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786267941944493-20982-0/minicluster-data/master-0/wal
uuid: "a761fd3aa90e40c68d587c1e199afdab"
format_stamp: "Formatted at 2026-08-09 09:36:38 on dist-test-slave-7kzw"
server_key: "6b9e3eabc0e8fdca7696ad624e8a188f"
server_key_iv: "ab9f0de3177e92a9261b233ec55c140a"
server_key_version: "encryptionkey@0"
I20260809 09:36:38.880944 14486 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786267941944493-20982-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786267941944493-20982-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786267941944493-20982-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 09:36:38.892616 14486 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 09:36:38.893287 14486 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 09:36:38.893496 14486 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 09:36:38.900538 14486 rpc_server.cc:307] RPC server started. Bound to: 127.20.125.190:45191
I20260809 09:36:38.900612 14581 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.125.190:45191 every 8 connection(s)
I20260809 09:36:38.901513 14486 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786267941944493-20982-0/minicluster-data/master-0/data/info.pb
I20260809 09:36:38.901937 20982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu as pid 14486
I20260809 09:36:38.902091 20982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786267941944493-20982-0/minicluster-data/master-0/wal/instance
I20260809 09:36:38.902443 20982 external_mini_cluster.cc:1428] Setting key 41b41481eac2d7e05cbc874864a032a5
I20260809 09:36:38.905004 14583 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 09:36:38.910650 14583 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a761fd3aa90e40c68d587c1e199afdab: Bootstrap starting.
I20260809 09:36:38.912083 14587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:36:38.903947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52132 (local address 127.20.125.190:45191)
0809 09:36:38.904640 (+ 693us) server_negotiation.cc:206] Beginning negotiation
0809 09:36:38.904655 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 09:36:38.904692 (+ 37us) server_negotiation.cc:407] Connection header received
0809 09:36:38.905421 (+ 729us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:36:38.905441 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:36:38.905825 (+ 384us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:36:38.906196 (+ 371us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:36:38.906695 (+ 499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:36:38.907682 (+ 987us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:36:38.908633 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:36:38.909109 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:36:38.909287 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:36:38.910269 (+ 982us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:36:38.910273 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:36:38.910285 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 09:36:38.910769 (+ 484us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:36:38.910840 (+ 71us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 09:36:38.910863 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:36:38.910978 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0809 09:36:38.911138 (+ 160us) server_negotiation.cc:299] Negotiation successful
0809 09:36:38.911391 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":344,"thread_start_us":168,"threads_started":1}
I20260809 09:36:38.912963 14583 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a761fd3aa90e40c68d587c1e199afdab: Neither blocks nor log segments found. Creating new log.
I20260809 09:36:38.913749 14583 log.cc:824] T 00000000000000000000000000000000 P a761fd3aa90e40c68d587c1e199afdab: Log is configured to *not* fsync() on all Append() calls
I20260809 09:36:38.915817 14583 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a761fd3aa90e40c68d587c1e199afdab: No bootstrap required, opened a new log
I20260809 09:36:38.919476 14583 raft_consensus.cc:348] T 00000000000000000000000000000000 P a761fd3aa90e40c68d587c1e199afdab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a761fd3aa90e40c68d587c1e199afdab" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 45191 } }
I20260809 09:36:38.919682 14583 raft_consensus.cc:374] T 00000000000000000000000000000000 P a761fd3aa90e40c68d587c1e199afdab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 09:36:38.919720 14583 raft_consensus.cc:729] T 00000000000000000000000000000000 P a761fd3aa90e40c68d587c1e199afdab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a761fd3aa90e40c68d587c1e199afdab, State: Initialized, Role: FOLLOWER
I20260809 09:36:38.920135 14583 consensus_queue.cc:260] T 00000000000000000000000000000000 P a761fd3aa90e40c68d587c1e199afdab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a761fd3aa90e40c68d587c1e199afdab" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 45191 } }
I20260809 09:36:38.920288 14583 raft_consensus.cc:388] T 00000000000000000000000000000000 P a761fd3aa90e40c68d587c1e199afdab [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 09:36:38.920375 14583 raft_consensus.cc:482] T 00000000000000000000000000000000 P a761fd3aa90e40c68d587c1e199afdab [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 09:36:38.920492 14583 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a761fd3aa90e40c68d587c1e199afdab [term 0 FOLLOWER]: Advancing to term 1
I20260809 09:36:38.921389 14583 raft_consensus.cc:504] T 00000000000000000000000000000000 P a761fd3aa90e40c68d587c1e199afdab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a761fd3aa90e40c68d587c1e199afdab" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 45191 } }
I20260809 09:36:38.921741 14583 leader_election.cc:302] T 00000000000000000000000000000000 P a761fd3aa90e40c68d587c1e199afdab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a761fd3aa90e40c68d587c1e199afdab; no voters:
I20260809 09:36:38.922029 14583 leader_election.cc:288] T 00000000000000000000000000000000 P a761fd3aa90e40c68d587c1e199afdab [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 09:36:38.922173 14589 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a761fd3aa90e40c68d587c1e199afdab [term 1 FOLLOWER]: Leader election won for term 1
I20260809 09:36:38.922423 14589 raft_consensus.cc:686] T 00000000000000000000000000000000 P a761fd3aa90e40c68d587c1e199afdab [term 1 LEADER]: Becoming Leader. State: Replica: a761fd3aa90e40c68d587c1e199afdab, State: Running, Role: LEADER
I20260809 09:36:38.922830 14589 consensus_queue.cc:237] T 00000000000000000000000000000000 P a761fd3aa90e40c68d587c1e199afdab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a761fd3aa90e40c68d587c1e199afdab" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 45191 } }
I20260809 09:36:38.923372 14583 sys_catalog.cc:564] T 00000000000000000000000000000000 P a761fd3aa90e40c68d587c1e199afdab [sys.catalog]: configured and running, proceeding with master startup.
I20260809 09:36:38.924616 14591 sys_catalog.cc:455] T 00000000000000000000000000000000 P a761fd3aa90e40c68d587c1e199afdab [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a761fd3aa90e40c68d587c1e199afdab" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a761fd3aa90e40c68d587c1e199afdab" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 45191 } } }
I20260809 09:36:38.924661 14592 sys_catalog.cc:455] T 00000000000000000000000000000000 P a761fd3aa90e40c68d587c1e199afdab [sys.catalog]: SysCatalogTable state changed. Reason: New leader a761fd3aa90e40c68d587c1e199afdab. Latest consensus state: current_term: 1 leader_uuid: "a761fd3aa90e40c68d587c1e199afdab" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a761fd3aa90e40c68d587c1e199afdab" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 45191 } } }
I20260809 09:36:38.924746 14591 sys_catalog.cc:458] T 00000000000000000000000000000000 P a761fd3aa90e40c68d587c1e199afdab [sys.catalog]: This master's current role is: LEADER
I20260809 09:36:38.924782 14592 sys_catalog.cc:458] T 00000000000000000000000000000000 P a761fd3aa90e40c68d587c1e199afdab [sys.catalog]: This master's current role is: LEADER
I20260809 09:36:38.925143 14601 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 09:36:38.928509 14601 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 09:36:38.934860 14601 catalog_manager.cc:1347] Generated new cluster ID: 236eb15c57b64811848a9781c613c324
I20260809 09:36:38.934983 14601 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 09:36:38.965114 14601 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 09:36:38.966123 14601 catalog_manager.cc:1504] Loading token signing keys...
I20260809 09:36:38.972015 14601 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a761fd3aa90e40c68d587c1e199afdab: Generated new TSK 0
I20260809 09:36:38.972805 14601 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 09:36:38.976418 20982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
/tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786267941944493-20982-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786267941944493-20982-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786267941944493-20982-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786267941944493-20982-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.125.129:0
--local_ip_for_outbound_sockets=127.20.125.129
--webserver_interface=127.20.125.129
--webserver_port=0
--tserver_master_addrs=127.20.125.190:45191
--builtin_ntp_servers=127.20.125.148:38147
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 09:36:39.095136 14616 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 09:36:39.095376 14616 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 09:36:39.095412 14616 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 09:36:39.099355 14616 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 09:36:39.099403 14616 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 09:36:39.099454 14616 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.125.129
I20260809 09:36:39.104123 14616 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.125.148:38147
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786267941944493-20982-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786267941944493-20982-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.125.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786267941944493-20982-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.125.129
--webserver_port=0
--tserver_master_addrs=127.20.125.190:45191
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14616
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.125.129
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786267941944493-20982-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 08:38:55 UTC on ccycloud.quasar-njjleu.root.comops.site
build id 4575935
I20260809 09:36:39.105156 14616 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 09:36:39.106575 14616 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 09:36:39.116783 14623 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:36:39.116848 14622 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:36:39.116783 14625 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:36:39.117463 14616 server_base.cc:1034] running on GCE node
I20260809 09:36:39.117965 14616 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 09:36:39.118736 14616 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 09:36:39.119905 14616 hybrid_clock.cc:648] HybridClock initialized: now 1786268199119887 us; error 30 us; skew 500 ppm
I20260809 09:36:39.121795 14616 webserver.cc:460] Webserver started at http://127.20.125.129:43187/ using document root <none> and password file <none>
I20260809 09:36:39.122391 14616 fs_manager.cc:359] Metadata directory not provided
I20260809 09:36:39.122457 14616 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 09:36:39.122753 14616 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 09:36:39.124958 14616 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786267941944493-20982-0/minicluster-data/ts-0/data/instance:
uuid: "9ea20a89428d477b81ce6458c5648c3d"
format_stamp: "Formatted at 2026-08-09 09:36:39 on dist-test-slave-7kzw"
server_key: "b683ccd09d3e5ca12add87ca76b6da53"
server_key_iv: "58b2ec2cb21f97662e7e91ade2796e77"
server_key_version: "encryptionkey@0"
I20260809 09:36:39.125531 14616 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786267941944493-20982-0/minicluster-data/ts-0/wal/instance:
uuid: "9ea20a89428d477b81ce6458c5648c3d"
format_stamp: "Formatted at 2026-08-09 09:36:39 on dist-test-slave-7kzw"
server_key: "b683ccd09d3e5ca12add87ca76b6da53"
server_key_iv: "58b2ec2cb21f97662e7e91ade2796e77"
server_key_version: "encryptionkey@0"
I20260809 09:36:39.129410 14616 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.003s
I20260809 09:36:39.132131 14632 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 09:36:39.133569 14616 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 09:36:39.133735 14616 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786267941944493-20982-0/minicluster-data/ts-0/data,/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786267941944493-20982-0/minicluster-data/ts-0/wal
uuid: "9ea20a89428d477b81ce6458c5648c3d"
format_stamp: "Formatted at 2026-08-09 09:36:39 on dist-test-slave-7kzw"
server_key: "b683ccd09d3e5ca12add87ca76b6da53"
server_key_iv: "58b2ec2cb21f97662e7e91ade2796e77"
server_key_version: "encryptionkey@0"
I20260809 09:36:39.133839 14616 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786267941944493-20982-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786267941944493-20982-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786267941944493-20982-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 09:36:39.151135 14616 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 09:36:39.151840 14616 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 09:36:39.152066 14616 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 09:36:39.152700 14616 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 09:36:39.153712 14616 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 09:36:39.153766 14616 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 09:36:39.153844 14616 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 09:36:39.153883 14616 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 09:36:39.163628 14616 rpc_server.cc:307] RPC server started. Bound to: 127.20.125.129:35757
I20260809 09:36:39.163710 14810 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.125.129:35757 every 8 connection(s)
I20260809 09:36:39.164698 14616 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786267941944493-20982-0/minicluster-data/ts-0/data/info.pb
I20260809 09:36:39.174252 14587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:36:39.166578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.129:38285 (local address 127.20.125.190:45191)
0809 09:36:39.166723 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0809 09:36:39.166726 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 09:36:39.167906 (+ 1180us) server_negotiation.cc:407] Connection header received
0809 09:36:39.169367 (+ 1461us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:36:39.169370 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:36:39.169423 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:36:39.169512 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:36:39.170625 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:36:39.171218 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:36:39.172362 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:36:39.172577 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:36:39.172659 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:36:39.173723 (+ 1064us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:36:39.173726 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:36:39.173728 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 09:36:39.173741 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:36:39.173762 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 09:36:39.173770 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:36:39.173832 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0809 09:36:39.173981 (+ 149us) server_negotiation.cc:299] Negotiation successful
0809 09:36:39.174050 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260809 09:36:39.174376 20982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu as pid 14616
I20260809 09:36:39.174556 20982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786267941944493-20982-0/minicluster-data/ts-0/wal/instance
I20260809 09:36:39.174893 20982 external_mini_cluster.cc:1428] Setting key 9ca9e6fab714768b00f7ade05c9cf079
I20260809 09:36:39.175192 14815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:36:39.166965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:45191 (local address 127.20.125.129:38285)
0809 09:36:39.167699 (+ 734us) negotiation.cc:107] Waiting for socket to connect
0809 09:36:39.167754 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0809 09:36:39.169116 (+ 1362us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:36:39.169723 (+ 607us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:36:39.169730 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:36:39.169836 (+ 106us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:36:39.170310 (+ 474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:36:39.170323 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:36:39.171354 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:36:39.171357 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:36:39.172230 (+ 873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:36:39.172239 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:36:39.172712 (+ 473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:36:39.172715 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:36:39.172878 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:36:39.173458 (+ 580us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 09:36:39.173460 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:36:39.173498 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 09:36:39.173558 (+ 60us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 09:36:39.173848 (+ 290us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 09:36:39.173855 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 09:36:39.173860 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 09:36:39.173954 (+ 94us) client_negotiation.cc:240] Negotiation successful
0809 09:36:39.174177 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":269,"thread_start_us":108,"threads_started":1}
I20260809 09:36:39.176481 14811 heartbeater.cc:347] Connected to a master server at 127.20.125.190:45191
I20260809 09:36:39.176786 14811 heartbeater.cc:464] Registering TS with master...
I20260809 09:36:39.177265 20982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
/tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786267941944493-20982-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786267941944493-20982-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786267941944493-20982-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786267941944493-20982-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.125.130:0
--local_ip_for_outbound_sockets=127.20.125.130
--webserver_interface=127.20.125.130
--webserver_port=0
--tserver_master_addrs=127.20.125.190:45191
--builtin_ntp_servers=127.20.125.148:38147
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260809 09:36:39.177431 14811 heartbeater.cc:511] Master 127.20.125.190:45191 requested a full tablet report, sending...
I20260809 09:36:39.179180 14529 ts_manager.cc:194] Registered new tserver with Master: 9ea20a89428d477b81ce6458c5648c3d (127.20.125.129:35757)
I20260809 09:36:39.180795 14529 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.20.125.129:38285
W20260809 09:36:39.292466 14817 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 09:36:39.292717 14817 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 09:36:39.292761 14817 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 09:36:39.296705 14817 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 09:36:39.296758 14817 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 09:36:39.296809 14817 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.125.130
I20260809 09:36:39.301349 14817 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.125.148:38147
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786267941944493-20982-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786267941944493-20982-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.125.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786267941944493-20982-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.125.130
--webserver_port=0
--tserver_master_addrs=127.20.125.190:45191
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14817
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.125.130
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786267941944493-20982-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 08:38:55 UTC on ccycloud.quasar-njjleu.root.comops.site
build id 4575935
I20260809 09:36:39.302335 14817 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 09:36:39.303702 14817 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 09:36:39.313879 14824 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:36:39.313895 14831 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:36:39.313896 14825 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:36:39.314301 14817 server_base.cc:1034] running on GCE node
I20260809 09:36:39.314786 14817 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 09:36:39.315485 14817 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 09:36:39.316687 14817 hybrid_clock.cc:648] HybridClock initialized: now 1786268199316652 us; error 47 us; skew 500 ppm
I20260809 09:36:39.318615 14817 webserver.cc:460] Webserver started at http://127.20.125.130:40395/ using document root <none> and password file <none>
I20260809 09:36:39.319219 14817 fs_manager.cc:359] Metadata directory not provided
I20260809 09:36:39.319327 14817 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 09:36:39.319569 14817 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 09:36:39.321641 14817 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786267941944493-20982-0/minicluster-data/ts-1/data/instance:
uuid: "93958f2262b64c7da41d9bd9487b970a"
format_stamp: "Formatted at 2026-08-09 09:36:39 on dist-test-slave-7kzw"
server_key: "5eb097fdfabb3bb143ac22ef7b744cd3"
server_key_iv: "0c26e0b0b5973d5384dabad44f6b1904"
server_key_version: "encryptionkey@0"
I20260809 09:36:39.322219 14817 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786267941944493-20982-0/minicluster-data/ts-1/wal/instance:
uuid: "93958f2262b64c7da41d9bd9487b970a"
format_stamp: "Formatted at 2026-08-09 09:36:39 on dist-test-slave-7kzw"
server_key: "5eb097fdfabb3bb143ac22ef7b744cd3"
server_key_iv: "0c26e0b0b5973d5384dabad44f6b1904"
server_key_version: "encryptionkey@0"
I20260809 09:36:39.326186 14817 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.003s
I20260809 09:36:39.328644 14840 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 09:36:39.329874 14817 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.001s
I20260809 09:36:39.330008 14817 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786267941944493-20982-0/minicluster-data/ts-1/data,/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786267941944493-20982-0/minicluster-data/ts-1/wal
uuid: "93958f2262b64c7da41d9bd9487b970a"
format_stamp: "Formatted at 2026-08-09 09:36:39 on dist-test-slave-7kzw"
server_key: "5eb097fdfabb3bb143ac22ef7b744cd3"
server_key_iv: "0c26e0b0b5973d5384dabad44f6b1904"
server_key_version: "encryptionkey@0"
I20260809 09:36:39.330116 14817 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786267941944493-20982-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786267941944493-20982-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786267941944493-20982-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 09:36:39.342540 14817 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 09:36:39.343158 14817 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 09:36:39.343362 14817 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 09:36:39.343920 14817 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 09:36:39.344813 14817 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 09:36:39.344897 14817 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 09:36:39.344991 14817 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 09:36:39.345043 14817 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 09:36:39.354152 14817 rpc_server.cc:307] RPC server started. Bound to: 127.20.125.130:44983
I20260809 09:36:39.354239 15005 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.125.130:44983 every 8 connection(s)
I20260809 09:36:39.355135 14817 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786267941944493-20982-0/minicluster-data/ts-1/data/info.pb
I20260809 09:36:39.355957 20982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu as pid 14817
I20260809 09:36:39.356096 20982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786267941944493-20982-0/minicluster-data/ts-1/wal/instance
I20260809 09:36:39.356401 20982 external_mini_cluster.cc:1428] Setting key 749abdd7d091119b698608c5515e66f9
I20260809 09:36:39.358835 20982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
/tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786267941944493-20982-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786267941944493-20982-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786267941944493-20982-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786267941944493-20982-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.125.131:0
--local_ip_for_outbound_sockets=127.20.125.131
--webserver_interface=127.20.125.131
--webserver_port=0
--tserver_master_addrs=127.20.125.190:45191
--builtin_ntp_servers=127.20.125.148:38147
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260809 09:36:39.365770 14587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:36:39.357294 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.130:60301 (local address 127.20.125.190:45191)
0809 09:36:39.357442 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0809 09:36:39.357446 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 09:36:39.358692 (+ 1246us) server_negotiation.cc:407] Connection header received
0809 09:36:39.360831 (+ 2139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:36:39.360834 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:36:39.360882 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:36:39.360961 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:36:39.361877 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:36:39.362409 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:36:39.363617 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:36:39.363857 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:36:39.363975 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:36:39.365218 (+ 1243us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:36:39.365221 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:36:39.365223 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 09:36:39.365237 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:36:39.365258 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 09:36:39.365266 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:36:39.365346 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0809 09:36:39.365521 (+ 175us) server_negotiation.cc:299] Negotiation successful
0809 09:36:39.365586 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260809 09:36:39.366694 15009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:36:39.357637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:45191 (local address 127.20.125.130:60301)
0809 09:36:39.358520 (+ 883us) negotiation.cc:107] Waiting for socket to connect
0809 09:36:39.358578 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0809 09:36:39.360570 (+ 1992us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:36:39.361137 (+ 567us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:36:39.361143 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:36:39.361233 (+ 90us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:36:39.361710 (+ 477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:36:39.361722 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:36:39.362596 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:36:39.362599 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:36:39.363483 (+ 884us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:36:39.363492 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:36:39.364018 (+ 526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:36:39.364022 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:36:39.364195 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:36:39.364916 (+ 721us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 09:36:39.364918 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:36:39.364959 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 09:36:39.365024 (+ 65us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 09:36:39.365349 (+ 325us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 09:36:39.365362 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 09:36:39.365368 (+ 6us) client_negotiation.cc:769] Sending connection context
0809 09:36:39.365505 (+ 137us) client_negotiation.cc:240] Negotiation successful
0809 09:36:39.365786 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":359,"thread_start_us":158,"threads_started":1}
I20260809 09:36:39.368008 15006 heartbeater.cc:347] Connected to a master server at 127.20.125.190:45191
I20260809 09:36:39.368299 15006 heartbeater.cc:464] Registering TS with master...
I20260809 09:36:39.368860 15006 heartbeater.cc:511] Master 127.20.125.190:45191 requested a full tablet report, sending...
I20260809 09:36:39.370075 14529 ts_manager.cc:194] Registered new tserver with Master: 93958f2262b64c7da41d9bd9487b970a (127.20.125.130:44983)
I20260809 09:36:39.371084 14529 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.20.125.130:60301
W20260809 09:36:39.475884 15010 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 09:36:39.476171 15010 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 09:36:39.476212 15010 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 09:36:39.480223 15010 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 09:36:39.480279 15010 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 09:36:39.480334 15010 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.125.131
I20260809 09:36:39.485215 15010 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.125.148:38147
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786267941944493-20982-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786267941944493-20982-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.125.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786267941944493-20982-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.125.131
--webserver_port=0
--tserver_master_addrs=127.20.125.190:45191
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15010
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.125.131
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786267941944493-20982-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 08:38:55 UTC on ccycloud.quasar-njjleu.root.comops.site
build id 4575935
I20260809 09:36:39.486315 15010 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 09:36:39.487805 15010 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 09:36:39.497771 15022 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:36:39.497849 15018 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:36:39.497766 15019 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:36:39.498010 15010 server_base.cc:1034] running on GCE node
I20260809 09:36:39.498520 15010 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 09:36:39.499174 15010 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 09:36:39.500372 15010 hybrid_clock.cc:648] HybridClock initialized: now 1786268199500359 us; error 34 us; skew 500 ppm
I20260809 09:36:39.502269 15010 webserver.cc:460] Webserver started at http://127.20.125.131:38301/ using document root <none> and password file <none>
I20260809 09:36:39.502936 15010 fs_manager.cc:359] Metadata directory not provided
I20260809 09:36:39.503034 15010 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 09:36:39.503262 15010 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 09:36:39.505288 15010 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786267941944493-20982-0/minicluster-data/ts-2/data/instance:
uuid: "f83294e89ecb4acebde75ecd689275bc"
format_stamp: "Formatted at 2026-08-09 09:36:39 on dist-test-slave-7kzw"
server_key: "729a822355e9a9e4168bbc3af27114f9"
server_key_iv: "5c2cf530f049e36cf22f941b552fdb19"
server_key_version: "encryptionkey@0"
I20260809 09:36:39.505869 15010 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786267941944493-20982-0/minicluster-data/ts-2/wal/instance:
uuid: "f83294e89ecb4acebde75ecd689275bc"
format_stamp: "Formatted at 2026-08-09 09:36:39 on dist-test-slave-7kzw"
server_key: "729a822355e9a9e4168bbc3af27114f9"
server_key_iv: "5c2cf530f049e36cf22f941b552fdb19"
server_key_version: "encryptionkey@0"
I20260809 09:36:39.509619 15010 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260809 09:36:39.512058 15028 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 09:36:39.513264 15010 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 09:36:39.513401 15010 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786267941944493-20982-0/minicluster-data/ts-2/data,/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786267941944493-20982-0/minicluster-data/ts-2/wal
uuid: "f83294e89ecb4acebde75ecd689275bc"
format_stamp: "Formatted at 2026-08-09 09:36:39 on dist-test-slave-7kzw"
server_key: "729a822355e9a9e4168bbc3af27114f9"
server_key_iv: "5c2cf530f049e36cf22f941b552fdb19"
server_key_version: "encryptionkey@0"
I20260809 09:36:39.513511 15010 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786267941944493-20982-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786267941944493-20982-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786267941944493-20982-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 09:36:39.540074 15010 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 09:36:39.540789 15010 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 09:36:39.541003 15010 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 09:36:39.541606 15010 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 09:36:39.542580 15010 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 09:36:39.542656 15010 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 09:36:39.542723 15010 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 09:36:39.542769 15010 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 09:36:39.551786 15010 rpc_server.cc:307] RPC server started. Bound to: 127.20.125.131:36735
I20260809 09:36:39.551824 15209 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.125.131:36735 every 8 connection(s)
I20260809 09:36:39.552662 15010 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786267941944493-20982-0/minicluster-data/ts-2/data/info.pb
I20260809 09:36:39.557178 20982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu as pid 15010
I20260809 09:36:39.557317 20982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786267941944493-20982-0/minicluster-data/ts-2/wal/instance
I20260809 09:36:39.557644 20982 external_mini_cluster.cc:1428] Setting key 58b0a8097fc383ce3ca19610d85b3ed3
I20260809 09:36:39.562318 14587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:36:39.554502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.131:36987 (local address 127.20.125.190:45191)
0809 09:36:39.554628 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0809 09:36:39.554632 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 09:36:39.555676 (+ 1044us) server_negotiation.cc:407] Connection header received
0809 09:36:39.557077 (+ 1401us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:36:39.557080 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:36:39.557137 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:36:39.557261 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:36:39.558257 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:36:39.559030 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:36:39.560461 (+ 1431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:36:39.560707 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:36:39.560793 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:36:39.561850 (+ 1057us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:36:39.561853 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:36:39.561855 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 09:36:39.561866 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:36:39.561884 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 09:36:39.561892 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:36:39.561953 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0809 09:36:39.562160 (+ 207us) server_negotiation.cc:299] Negotiation successful
0809 09:36:39.562210 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260809 09:36:39.563163 15213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:36:39.554891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:45191 (local address 127.20.125.131:36987)
0809 09:36:39.555529 (+ 638us) negotiation.cc:107] Waiting for socket to connect
0809 09:36:39.555569 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 09:36:39.556855 (+ 1286us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:36:39.557468 (+ 613us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:36:39.557475 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:36:39.557555 (+ 80us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:36:39.558063 (+ 508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:36:39.558076 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:36:39.559201 (+ 1125us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:36:39.559205 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:36:39.560334 (+ 1129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:36:39.560345 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:36:39.560851 (+ 506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:36:39.560854 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:36:39.561003 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:36:39.561587 (+ 584us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 09:36:39.561588 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:36:39.561629 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 09:36:39.561701 (+ 72us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 09:36:39.561994 (+ 293us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 09:36:39.562003 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 09:36:39.562008 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 09:36:39.562128 (+ 120us) client_negotiation.cc:240] Negotiation successful
0809 09:36:39.562343 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":107,"threads_started":1}
I20260809 09:36:39.564329 15210 heartbeater.cc:347] Connected to a master server at 127.20.125.190:45191
I20260809 09:36:39.564528 15210 heartbeater.cc:464] Registering TS with master...
I20260809 09:36:39.564961 15210 heartbeater.cc:511] Master 127.20.125.190:45191 requested a full tablet report, sending...
I20260809 09:36:39.565902 14529 ts_manager.cc:194] Registered new tserver with Master: f83294e89ecb4acebde75ecd689275bc (127.20.125.131:36735)
I20260809 09:36:39.566416 14529 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.20.125.131:36987
I20260809 09:36:39.571914 20982 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 09:36:39.578003 14587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:36:39.574058 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52136 (local address 127.20.125.190:45191)
0809 09:36:39.574205 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0809 09:36:39.574209 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 09:36:39.574380 (+ 171us) server_negotiation.cc:407] Connection header received
0809 09:36:39.574506 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:36:39.574511 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:36:39.574566 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:36:39.574649 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:36:39.574987 (+ 338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:36:39.575540 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:36:39.576388 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:36:39.576616 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:36:39.576718 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:36:39.577513 (+ 795us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:36:39.577516 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:36:39.577518 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 09:36:39.577529 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:36:39.577546 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 09:36:39.577554 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:36:39.577620 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0809 09:36:39.577845 (+ 225us) server_negotiation.cc:299] Negotiation successful
0809 09:36:39.577897 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260809 09:36:39.581118 14529 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:52136:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 09:36:39.584398 14529 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 09:36:39.599260 15227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:36:39.593720 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.130:44983 (local address 127.0.0.1:46092)
0809 09:36:39.594392 (+ 672us) negotiation.cc:107] Waiting for socket to connect
0809 09:36:39.594406 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 09:36:39.594525 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:36:39.595123 (+ 598us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:36:39.595126 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:36:39.595153 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 09:36:39.595302 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:36:39.595308 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:36:39.596829 (+ 1521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:36:39.596833 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:36:39.597657 (+ 824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:36:39.597666 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:36:39.598700 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:36:39.598704 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:36:39.598817 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:36:39.598860 (+ 43us) client_negotiation.cc:769] Sending connection context
0809 09:36:39.598964 (+ 104us) client_negotiation.cc:240] Negotiation successful
0809 09:36:39.599088 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":610,"mutex_wait_us":23,"spinlock_wait_cycles":5120,"thread_start_us":83,"threads_started":1}
I20260809 09:36:39.599534 15229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:36:39.593331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45736 (local address 127.20.125.131:36735)
0809 09:36:39.594535 (+ 1204us) server_negotiation.cc:206] Beginning negotiation
0809 09:36:39.594548 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 09:36:39.594571 (+ 23us) server_negotiation.cc:407] Connection header received
0809 09:36:39.594645 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:36:39.594650 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:36:39.594831 (+ 181us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:36:39.594979 (+ 148us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 09:36:39.595462 (+ 483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:36:39.596173 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:36:39.598063 (+ 1890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:36:39.598844 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:36:39.598971 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:36:39.599134 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0809 09:36:39.599272 (+ 138us) server_negotiation.cc:299] Negotiation successful
0809 09:36:39.599367 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1080,"thread_start_us":164,"threads_started":1}
I20260809 09:36:39.599867 15230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:36:39.593507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52502 (local address 127.20.125.129:35757)
0809 09:36:39.594363 (+ 856us) server_negotiation.cc:206] Beginning negotiation
0809 09:36:39.594376 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 09:36:39.594404 (+ 28us) server_negotiation.cc:407] Connection header received
0809 09:36:39.594515 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:36:39.594520 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:36:39.594718 (+ 198us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:36:39.594883 (+ 165us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 09:36:39.595157 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:36:39.596296 (+ 1139us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:36:39.598080 (+ 1784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:36:39.598701 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:36:39.598816 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:36:39.598980 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0809 09:36:39.599588 (+ 608us) server_negotiation.cc:299] Negotiation successful
0809 09:36:39.599693 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":741,"thread_start_us":127,"threads_started":1}
I20260809 09:36:39.600157 15232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:36:39.593892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46092 (local address 127.20.125.130:44983)
0809 09:36:39.594535 (+ 643us) server_negotiation.cc:206] Beginning negotiation
0809 09:36:39.594548 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 09:36:39.594571 (+ 23us) server_negotiation.cc:407] Connection header received
0809 09:36:39.594645 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:36:39.594650 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:36:39.594831 (+ 181us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:36:39.594979 (+ 148us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 09:36:39.595433 (+ 454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:36:39.596673 (+ 1240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:36:39.597793 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:36:39.598545 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:36:39.598667 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:36:39.598850 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0809 09:36:39.599457 (+ 607us) server_negotiation.cc:299] Negotiation successful
0809 09:36:39.600002 (+ 545us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":516,"thread_start_us":118,"threads_started":1}
I20260809 09:36:39.599479 15228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:36:39.593007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.129:35757 (local address 127.0.0.1:52502)
0809 09:36:39.594026 (+ 1019us) negotiation.cc:107] Waiting for socket to connect
0809 09:36:39.594078 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0809 09:36:39.594214 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:36:39.594875 (+ 661us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:36:39.594881 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:36:39.594904 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 09:36:39.595040 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:36:39.595046 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:36:39.596414 (+ 1368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:36:39.596417 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:36:39.597924 (+ 1507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:36:39.597936 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:36:39.599039 (+ 1103us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:36:39.599043 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:36:39.599145 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:36:39.599168 (+ 23us) client_negotiation.cc:769] Sending connection context
0809 09:36:39.599254 (+ 86us) client_negotiation.cc:240] Negotiation successful
0809 09:36:39.599331 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":848,"mutex_wait_us":143,"spinlock_wait_cycles":3712,"thread_start_us":103,"threads_started":1}
I20260809 09:36:39.599637 15231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:36:39.593112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.131:36735 (local address 127.0.0.1:45736)
0809 09:36:39.594026 (+ 914us) negotiation.cc:107] Waiting for socket to connect
0809 09:36:39.594078 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0809 09:36:39.594214 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:36:39.595123 (+ 909us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:36:39.595126 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:36:39.595148 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 09:36:39.595296 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:36:39.595303 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:36:39.596300 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:36:39.596306 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:36:39.597931 (+ 1625us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:36:39.597937 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:36:39.598970 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:36:39.598974 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:36:39.599093 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:36:39.599117 (+ 24us) client_negotiation.cc:769] Sending connection context
0809 09:36:39.599213 (+ 96us) client_negotiation.cc:240] Negotiation successful
0809 09:36:39.599290 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":830,"mutex_wait_us":243,"spinlock_wait_cycles":8320,"thread_start_us":132,"threads_started":1}
I20260809 09:36:39.602962 14710 tablet_service.cc:1467] Processing CreateTablet for tablet e65c77582d2f4c46b845f83409f5562a (DEFAULT_TABLE table=test-table [id=5e9450bec1e24220a6076df89e74c69b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 09:36:39.602967 14908 tablet_service.cc:1467] Processing CreateTablet for tablet e65c77582d2f4c46b845f83409f5562a (DEFAULT_TABLE table=test-table [id=5e9450bec1e24220a6076df89e74c69b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 09:36:39.604015 14908 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e65c77582d2f4c46b845f83409f5562a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 09:36:39.604015 14710 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e65c77582d2f4c46b845f83409f5562a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 09:36:39.605254 15095 tablet_service.cc:1467] Processing CreateTablet for tablet e65c77582d2f4c46b845f83409f5562a (DEFAULT_TABLE table=test-table [id=5e9450bec1e24220a6076df89e74c69b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 09:36:39.606262 15095 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e65c77582d2f4c46b845f83409f5562a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 09:36:39.610421 15233 tablet_bootstrap.cc:492] T e65c77582d2f4c46b845f83409f5562a P 93958f2262b64c7da41d9bd9487b970a: Bootstrap starting.
I20260809 09:36:39.611869 15234 tablet_bootstrap.cc:492] T e65c77582d2f4c46b845f83409f5562a P 9ea20a89428d477b81ce6458c5648c3d: Bootstrap starting.
I20260809 09:36:39.612864 15233 tablet_bootstrap.cc:654] T e65c77582d2f4c46b845f83409f5562a P 93958f2262b64c7da41d9bd9487b970a: Neither blocks nor log segments found. Creating new log.
I20260809 09:36:39.613165 15235 tablet_bootstrap.cc:492] T e65c77582d2f4c46b845f83409f5562a P f83294e89ecb4acebde75ecd689275bc: Bootstrap starting.
I20260809 09:36:39.613751 15233 log.cc:824] T e65c77582d2f4c46b845f83409f5562a P 93958f2262b64c7da41d9bd9487b970a: Log is configured to *not* fsync() on all Append() calls
I20260809 09:36:39.614022 15234 tablet_bootstrap.cc:654] T e65c77582d2f4c46b845f83409f5562a P 9ea20a89428d477b81ce6458c5648c3d: Neither blocks nor log segments found. Creating new log.
I20260809 09:36:39.614897 15234 log.cc:824] T e65c77582d2f4c46b845f83409f5562a P 9ea20a89428d477b81ce6458c5648c3d: Log is configured to *not* fsync() on all Append() calls
I20260809 09:36:39.615568 15235 tablet_bootstrap.cc:654] T e65c77582d2f4c46b845f83409f5562a P f83294e89ecb4acebde75ecd689275bc: Neither blocks nor log segments found. Creating new log.
I20260809 09:36:39.616101 15233 tablet_bootstrap.cc:492] T e65c77582d2f4c46b845f83409f5562a P 93958f2262b64c7da41d9bd9487b970a: No bootstrap required, opened a new log
I20260809 09:36:39.616307 15233 ts_tablet_manager.cc:1397] T e65c77582d2f4c46b845f83409f5562a P 93958f2262b64c7da41d9bd9487b970a: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260809 09:36:39.616385 15235 log.cc:824] T e65c77582d2f4c46b845f83409f5562a P f83294e89ecb4acebde75ecd689275bc: Log is configured to *not* fsync() on all Append() calls
I20260809 09:36:39.617065 15234 tablet_bootstrap.cc:492] T e65c77582d2f4c46b845f83409f5562a P 9ea20a89428d477b81ce6458c5648c3d: No bootstrap required, opened a new log
I20260809 09:36:39.617271 15234 ts_tablet_manager.cc:1397] T e65c77582d2f4c46b845f83409f5562a P 9ea20a89428d477b81ce6458c5648c3d: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260809 09:36:39.618453 15235 tablet_bootstrap.cc:492] T e65c77582d2f4c46b845f83409f5562a P f83294e89ecb4acebde75ecd689275bc: No bootstrap required, opened a new log
I20260809 09:36:39.618652 15235 ts_tablet_manager.cc:1397] T e65c77582d2f4c46b845f83409f5562a P f83294e89ecb4acebde75ecd689275bc: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.001s
I20260809 09:36:39.620565 15234 raft_consensus.cc:348] T e65c77582d2f4c46b845f83409f5562a P 9ea20a89428d477b81ce6458c5648c3d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ea20a89428d477b81ce6458c5648c3d" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 35757 } } peers { permanent_uuid: "f83294e89ecb4acebde75ecd689275bc" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 36735 } } peers { permanent_uuid: "93958f2262b64c7da41d9bd9487b970a" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 44983 } }
I20260809 09:36:39.620815 15234 raft_consensus.cc:374] T e65c77582d2f4c46b845f83409f5562a P 9ea20a89428d477b81ce6458c5648c3d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 09:36:39.620867 15234 raft_consensus.cc:729] T e65c77582d2f4c46b845f83409f5562a P 9ea20a89428d477b81ce6458c5648c3d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ea20a89428d477b81ce6458c5648c3d, State: Initialized, Role: FOLLOWER
I20260809 09:36:39.621310 15234 consensus_queue.cc:260] T e65c77582d2f4c46b845f83409f5562a P 9ea20a89428d477b81ce6458c5648c3d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ea20a89428d477b81ce6458c5648c3d" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 35757 } } peers { permanent_uuid: "f83294e89ecb4acebde75ecd689275bc" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 36735 } } peers { permanent_uuid: "93958f2262b64c7da41d9bd9487b970a" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 44983 } }
I20260809 09:36:39.621968 14811 heartbeater.cc:503] Master 127.20.125.190:45191 was elected leader, sending a full tablet report...
I20260809 09:36:39.622290 15234 ts_tablet_manager.cc:1428] T e65c77582d2f4c46b845f83409f5562a P 9ea20a89428d477b81ce6458c5648c3d: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260809 09:36:39.621291 15233 raft_consensus.cc:348] T e65c77582d2f4c46b845f83409f5562a P 93958f2262b64c7da41d9bd9487b970a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ea20a89428d477b81ce6458c5648c3d" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 35757 } } peers { permanent_uuid: "f83294e89ecb4acebde75ecd689275bc" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 36735 } } peers { permanent_uuid: "93958f2262b64c7da41d9bd9487b970a" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 44983 } }
I20260809 09:36:39.622671 15233 raft_consensus.cc:374] T e65c77582d2f4c46b845f83409f5562a P 93958f2262b64c7da41d9bd9487b970a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 09:36:39.622767 15233 raft_consensus.cc:729] T e65c77582d2f4c46b845f83409f5562a P 93958f2262b64c7da41d9bd9487b970a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 93958f2262b64c7da41d9bd9487b970a, State: Initialized, Role: FOLLOWER
I20260809 09:36:39.622685 15235 raft_consensus.cc:348] T e65c77582d2f4c46b845f83409f5562a P f83294e89ecb4acebde75ecd689275bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ea20a89428d477b81ce6458c5648c3d" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 35757 } } peers { permanent_uuid: "f83294e89ecb4acebde75ecd689275bc" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 36735 } } peers { permanent_uuid: "93958f2262b64c7da41d9bd9487b970a" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 44983 } }
I20260809 09:36:39.622921 15235 raft_consensus.cc:374] T e65c77582d2f4c46b845f83409f5562a P f83294e89ecb4acebde75ecd689275bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 09:36:39.623080 15235 raft_consensus.cc:729] T e65c77582d2f4c46b845f83409f5562a P f83294e89ecb4acebde75ecd689275bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f83294e89ecb4acebde75ecd689275bc, State: Initialized, Role: FOLLOWER
I20260809 09:36:39.623261 15233 consensus_queue.cc:260] T e65c77582d2f4c46b845f83409f5562a P 93958f2262b64c7da41d9bd9487b970a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ea20a89428d477b81ce6458c5648c3d" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 35757 } } peers { permanent_uuid: "f83294e89ecb4acebde75ecd689275bc" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 36735 } } peers { permanent_uuid: "93958f2262b64c7da41d9bd9487b970a" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 44983 } }
I20260809 09:36:39.623531 15235 consensus_queue.cc:260] T e65c77582d2f4c46b845f83409f5562a P f83294e89ecb4acebde75ecd689275bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ea20a89428d477b81ce6458c5648c3d" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 35757 } } peers { permanent_uuid: "f83294e89ecb4acebde75ecd689275bc" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 36735 } } peers { permanent_uuid: "93958f2262b64c7da41d9bd9487b970a" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 44983 } }
I20260809 09:36:39.624024 15006 heartbeater.cc:503] Master 127.20.125.190:45191 was elected leader, sending a full tablet report...
I20260809 09:36:39.624325 15233 ts_tablet_manager.cc:1428] T e65c77582d2f4c46b845f83409f5562a P 93958f2262b64c7da41d9bd9487b970a: Time spent starting tablet: real 0.008s user 0.008s sys 0.000s
I20260809 09:36:39.625643 15235 ts_tablet_manager.cc:1428] T e65c77582d2f4c46b845f83409f5562a P f83294e89ecb4acebde75ecd689275bc: Time spent starting tablet: real 0.007s user 0.003s sys 0.002s
I20260809 09:36:39.626058 15210 heartbeater.cc:503] Master 127.20.125.190:45191 was elected leader, sending a full tablet report...
I20260809 09:36:39.645260 15243 raft_consensus.cc:482] T e65c77582d2f4c46b845f83409f5562a P f83294e89ecb4acebde75ecd689275bc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 09:36:39.645442 15243 raft_consensus.cc:504] T e65c77582d2f4c46b845f83409f5562a P f83294e89ecb4acebde75ecd689275bc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ea20a89428d477b81ce6458c5648c3d" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 35757 } } peers { permanent_uuid: "f83294e89ecb4acebde75ecd689275bc" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 36735 } } peers { permanent_uuid: "93958f2262b64c7da41d9bd9487b970a" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 44983 } }
I20260809 09:36:39.646833 15243 leader_election.cc:288] T e65c77582d2f4c46b845f83409f5562a P f83294e89ecb4acebde75ecd689275bc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9ea20a89428d477b81ce6458c5648c3d (127.20.125.129:35757), 93958f2262b64c7da41d9bd9487b970a (127.20.125.130:44983)
I20260809 09:36:39.650547 15213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:36:39.646802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.129:35757 (local address 127.20.125.131:35231)
0809 09:36:39.647007 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0809 09:36:39.647023 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 09:36:39.647111 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:36:39.647481 (+ 370us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:36:39.647487 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:36:39.647519 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 09:36:39.647728 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:36:39.647738 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:36:39.648568 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:36:39.648574 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:36:39.649459 (+ 885us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:36:39.649469 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:36:39.650092 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:36:39.650099 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:36:39.650195 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:36:39.650226 (+ 31us) client_negotiation.cc:769] Sending connection context
0809 09:36:39.650296 (+ 70us) client_negotiation.cc:240] Negotiation successful
0809 09:36:39.650386 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"mutex_wait_us":53}
I20260809 09:36:39.650571 15230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:36:39.646992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.131:35231 (local address 127.20.125.129:35757)
0809 09:36:39.647102 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0809 09:36:39.647107 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 09:36:39.647123 (+ 16us) server_negotiation.cc:407] Connection header received
0809 09:36:39.647230 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:36:39.647232 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:36:39.647301 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:36:39.647398 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 09:36:39.647876 (+ 478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:36:39.648428 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:36:39.649589 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:36:39.649962 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:36:39.650054 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:36:39.650123 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0809 09:36:39.650345 (+ 222us) server_negotiation.cc:299] Negotiation successful
0809 09:36:39.650420 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260809 09:36:39.650597 15232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:36:39.646907 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.131:51167 (local address 127.20.125.130:44983)
0809 09:36:39.647037 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0809 09:36:39.647042 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 09:36:39.647449 (+ 407us) server_negotiation.cc:407] Connection header received
0809 09:36:39.647512 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:36:39.647515 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:36:39.647568 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:36:39.647701 (+ 133us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 09:36:39.647985 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:36:39.648532 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:36:39.649624 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:36:39.649999 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:36:39.650146 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:36:39.650195 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0809 09:36:39.650395 (+ 200us) server_negotiation.cc:299] Negotiation successful
0809 09:36:39.650488 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260809 09:36:39.650599 15244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:36:39.646802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.130:44983 (local address 127.20.125.131:51167)
0809 09:36:39.647310 (+ 508us) negotiation.cc:107] Waiting for socket to connect
0809 09:36:39.647326 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 09:36:39.647426 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:36:39.647697 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:36:39.647700 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:36:39.647715 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 09:36:39.647844 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:36:39.647852 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:36:39.648643 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:36:39.648645 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:36:39.649469 (+ 824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:36:39.649478 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:36:39.650160 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:36:39.650169 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:36:39.650245 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:36:39.650263 (+ 18us) client_negotiation.cc:769] Sending connection context
0809 09:36:39.650361 (+ 98us) client_negotiation.cc:240] Negotiation successful
0809 09:36:39.650431 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":418,"mutex_wait_us":54,"thread_start_us":132,"threads_started":1}
I20260809 09:36:39.651211 14744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e65c77582d2f4c46b845f83409f5562a" candidate_uuid: "f83294e89ecb4acebde75ecd689275bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9ea20a89428d477b81ce6458c5648c3d" is_pre_election: true
I20260809 09:36:39.651384 14934 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e65c77582d2f4c46b845f83409f5562a" candidate_uuid: "f83294e89ecb4acebde75ecd689275bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93958f2262b64c7da41d9bd9487b970a" is_pre_election: true
I20260809 09:36:39.651535 14744 raft_consensus.cc:2454] T e65c77582d2f4c46b845f83409f5562a P 9ea20a89428d477b81ce6458c5648c3d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f83294e89ecb4acebde75ecd689275bc in term 0.
I20260809 09:36:39.651640 14934 raft_consensus.cc:2454] T e65c77582d2f4c46b845f83409f5562a P 93958f2262b64c7da41d9bd9487b970a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f83294e89ecb4acebde75ecd689275bc in term 0.
I20260809 09:36:39.652117 15029 leader_election.cc:302] T e65c77582d2f4c46b845f83409f5562a P f83294e89ecb4acebde75ecd689275bc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 93958f2262b64c7da41d9bd9487b970a, 9ea20a89428d477b81ce6458c5648c3d, f83294e89ecb4acebde75ecd689275bc; no voters:
I20260809 09:36:39.652405 15243 raft_consensus.cc:2781] T e65c77582d2f4c46b845f83409f5562a P f83294e89ecb4acebde75ecd689275bc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 09:36:39.652493 15243 raft_consensus.cc:482] T e65c77582d2f4c46b845f83409f5562a P f83294e89ecb4acebde75ecd689275bc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 09:36:39.652539 15243 raft_consensus.cc:3037] T e65c77582d2f4c46b845f83409f5562a P f83294e89ecb4acebde75ecd689275bc [term 0 FOLLOWER]: Advancing to term 1
I20260809 09:36:39.653709 15243 raft_consensus.cc:504] T e65c77582d2f4c46b845f83409f5562a P f83294e89ecb4acebde75ecd689275bc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ea20a89428d477b81ce6458c5648c3d" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 35757 } } peers { permanent_uuid: "f83294e89ecb4acebde75ecd689275bc" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 36735 } } peers { permanent_uuid: "93958f2262b64c7da41d9bd9487b970a" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 44983 } }
I20260809 09:36:39.654073 15243 leader_election.cc:288] T e65c77582d2f4c46b845f83409f5562a P f83294e89ecb4acebde75ecd689275bc [CANDIDATE]: Term 1 election: Requested vote from peers 9ea20a89428d477b81ce6458c5648c3d (127.20.125.129:35757), 93958f2262b64c7da41d9bd9487b970a (127.20.125.130:44983)
I20260809 09:36:39.654445 14934 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e65c77582d2f4c46b845f83409f5562a" candidate_uuid: "f83294e89ecb4acebde75ecd689275bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93958f2262b64c7da41d9bd9487b970a"
I20260809 09:36:39.654512 14744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e65c77582d2f4c46b845f83409f5562a" candidate_uuid: "f83294e89ecb4acebde75ecd689275bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9ea20a89428d477b81ce6458c5648c3d"
I20260809 09:36:39.654637 14744 raft_consensus.cc:3037] T e65c77582d2f4c46b845f83409f5562a P 9ea20a89428d477b81ce6458c5648c3d [term 0 FOLLOWER]: Advancing to term 1
I20260809 09:36:39.654635 14934 raft_consensus.cc:3037] T e65c77582d2f4c46b845f83409f5562a P 93958f2262b64c7da41d9bd9487b970a [term 0 FOLLOWER]: Advancing to term 1
I20260809 09:36:39.655666 14744 raft_consensus.cc:2454] T e65c77582d2f4c46b845f83409f5562a P 9ea20a89428d477b81ce6458c5648c3d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f83294e89ecb4acebde75ecd689275bc in term 1.
I20260809 09:36:39.655792 14934 raft_consensus.cc:2454] T e65c77582d2f4c46b845f83409f5562a P 93958f2262b64c7da41d9bd9487b970a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f83294e89ecb4acebde75ecd689275bc in term 1.
I20260809 09:36:39.655984 15033 leader_election.cc:302] T e65c77582d2f4c46b845f83409f5562a P f83294e89ecb4acebde75ecd689275bc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9ea20a89428d477b81ce6458c5648c3d, f83294e89ecb4acebde75ecd689275bc; no voters:
I20260809 09:36:39.656157 15243 raft_consensus.cc:2781] T e65c77582d2f4c46b845f83409f5562a P f83294e89ecb4acebde75ecd689275bc [term 1 FOLLOWER]: Leader election won for term 1
I20260809 09:36:39.656376 15243 raft_consensus.cc:686] T e65c77582d2f4c46b845f83409f5562a P f83294e89ecb4acebde75ecd689275bc [term 1 LEADER]: Becoming Leader. State: Replica: f83294e89ecb4acebde75ecd689275bc, State: Running, Role: LEADER
I20260809 09:36:39.656713 15243 consensus_queue.cc:237] T e65c77582d2f4c46b845f83409f5562a P f83294e89ecb4acebde75ecd689275bc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ea20a89428d477b81ce6458c5648c3d" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 35757 } } peers { permanent_uuid: "f83294e89ecb4acebde75ecd689275bc" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 36735 } } peers { permanent_uuid: "93958f2262b64c7da41d9bd9487b970a" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 44983 } }
I20260809 09:36:39.659833 14528 catalog_manager.cc:5495] T e65c77582d2f4c46b845f83409f5562a P f83294e89ecb4acebde75ecd689275bc reported cstate change: term changed from 0 to 1, leader changed from <none> to f83294e89ecb4acebde75ecd689275bc (127.20.125.131). New cstate: current_term: 1 leader_uuid: "f83294e89ecb4acebde75ecd689275bc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ea20a89428d477b81ce6458c5648c3d" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 35757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f83294e89ecb4acebde75ecd689275bc" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 36735 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "93958f2262b64c7da41d9bd9487b970a" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 44983 } health_report { overall_health: UNKNOWN } } }
W20260809 09:36:39.665885 14812 tablet.cc:2369] T e65c77582d2f4c46b845f83409f5562a P 9ea20a89428d477b81ce6458c5648c3d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 09:36:39.676883 15229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:36:39.673232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45740 (local address 127.20.125.131:36735)
0809 09:36:39.673381 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0809 09:36:39.673385 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 09:36:39.673399 (+ 14us) server_negotiation.cc:407] Connection header received
0809 09:36:39.673576 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:36:39.673578 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:36:39.673629 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:36:39.673779 (+ 150us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 09:36:39.674013 (+ 234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:36:39.674570 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:36:39.675417 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:36:39.675639 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:36:39.675761 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:36:39.676074 (+ 313us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 09:36:39.676484 (+ 410us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 09:36:39.676623 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0809 09:36:39.676670 (+ 47us) server_negotiation.cc:299] Negotiation successful
0809 09:36:39.676733 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 09:36:39.683224 14934 raft_consensus.cc:1261] T e65c77582d2f4c46b845f83409f5562a P 93958f2262b64c7da41d9bd9487b970a [term 1 FOLLOWER]: Refusing update from remote peer f83294e89ecb4acebde75ecd689275bc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 09:36:39.683290 14744 raft_consensus.cc:1261] T e65c77582d2f4c46b845f83409f5562a P 9ea20a89428d477b81ce6458c5648c3d [term 1 FOLLOWER]: Refusing update from remote peer f83294e89ecb4acebde75ecd689275bc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 09:36:39.683962 15243 consensus_queue.cc:1035] T e65c77582d2f4c46b845f83409f5562a P f83294e89ecb4acebde75ecd689275bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "93958f2262b64c7da41d9bd9487b970a" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 44983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 09:36:39.684191 15246 consensus_queue.cc:1035] T e65c77582d2f4c46b845f83409f5562a P f83294e89ecb4acebde75ecd689275bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "9ea20a89428d477b81ce6458c5648c3d" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 35757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 09:36:39.702417 14529 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:52136:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 09:36:39.702682 14529 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:52136:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 09:36:39.705461 14529 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a761fd3aa90e40c68d587c1e199afdab: Sending DeleteTablet for 3 replicas of tablet e65c77582d2f4c46b845f83409f5562a
I20260809 09:36:39.706329 14710 tablet_service.cc:1514] Processing DeleteTablet for tablet e65c77582d2f4c46b845f83409f5562a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 09:36:39 UTC) from {username='slave'} at 127.0.0.1:52502
I20260809 09:36:39.706354 15095 tablet_service.cc:1514] Processing DeleteTablet for tablet e65c77582d2f4c46b845f83409f5562a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 09:36:39 UTC) from {username='slave'} at 127.0.0.1:45736
I20260809 09:36:39.706377 14908 tablet_service.cc:1514] Processing DeleteTablet for tablet e65c77582d2f4c46b845f83409f5562a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 09:36:39 UTC) from {username='slave'} at 127.0.0.1:46092
I20260809 09:36:39.706967 20982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu with pid 14616
I20260809 09:36:39.707037 15272 tablet_replica.cc:330] T e65c77582d2f4c46b845f83409f5562a P 9ea20a89428d477b81ce6458c5648c3d: stopping tablet replica
I20260809 09:36:39.717217 15270 tablet_replica.cc:330] T e65c77582d2f4c46b845f83409f5562a P 93958f2262b64c7da41d9bd9487b970a: stopping tablet replica
I20260809 09:36:39.717965 15270 raft_consensus.cc:2229] T e65c77582d2f4c46b845f83409f5562a P 93958f2262b64c7da41d9bd9487b970a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 09:36:39.718372 15270 raft_consensus.cc:2258] T e65c77582d2f4c46b845f83409f5562a P 93958f2262b64c7da41d9bd9487b970a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 09:36:39.718806 15269 tablet_replica.cc:330] T e65c77582d2f4c46b845f83409f5562a P f83294e89ecb4acebde75ecd689275bc: stopping tablet replica
I20260809 09:36:39.719127 15269 raft_consensus.cc:2229] T e65c77582d2f4c46b845f83409f5562a P f83294e89ecb4acebde75ecd689275bc [term 1 LEADER]: Raft consensus shutting down.
I20260809 09:36:39.719525 15269 raft_consensus.cc:2258] T e65c77582d2f4c46b845f83409f5562a P f83294e89ecb4acebde75ecd689275bc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 09:36:39.721140 15269 ts_tablet_manager.cc:1905] T e65c77582d2f4c46b845f83409f5562a P f83294e89ecb4acebde75ecd689275bc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 09:36:39.721140 15270 ts_tablet_manager.cc:1905] T e65c77582d2f4c46b845f83409f5562a P 93958f2262b64c7da41d9bd9487b970a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 09:36:39.724014 15270 ts_tablet_manager.cc:1918] T e65c77582d2f4c46b845f83409f5562a P 93958f2262b64c7da41d9bd9487b970a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 09:36:39.724162 15270 log.cc:1196] T e65c77582d2f4c46b845f83409f5562a P 93958f2262b64c7da41d9bd9487b970a: Deleting WAL directory at /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786267941944493-20982-0/minicluster-data/ts-1/wal/wals/e65c77582d2f4c46b845f83409f5562a
I20260809 09:36:39.724493 15270 ts_tablet_manager.cc:1939] T e65c77582d2f4c46b845f83409f5562a P 93958f2262b64c7da41d9bd9487b970a: Deleting consensus metadata
W20260809 09:36:39.725612 14511 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.20.125.129:35757 (error 108)
I20260809 09:36:39.725836 14508 catalog_manager.cc:4841] TS 93958f2262b64c7da41d9bd9487b970a (127.20.125.130:44983): tablet e65c77582d2f4c46b845f83409f5562a (table test-table [id=5e9450bec1e24220a6076df89e74c69b]) successfully deleted
I20260809 09:36:39.725888 20982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu with pid 14817
I20260809 09:36:39.725198 15269 ts_tablet_manager.cc:1918] T e65c77582d2f4c46b845f83409f5562a P f83294e89ecb4acebde75ecd689275bc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 09:36:39.729831 15269 log.cc:1196] T e65c77582d2f4c46b845f83409f5562a P f83294e89ecb4acebde75ecd689275bc: Deleting WAL directory at /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786267941944493-20982-0/minicluster-data/ts-2/wal/wals/e65c77582d2f4c46b845f83409f5562a
I20260809 09:36:39.730254 15269 ts_tablet_manager.cc:1939] T e65c77582d2f4c46b845f83409f5562a P f83294e89ecb4acebde75ecd689275bc: Deleting consensus metadata
I20260809 09:36:39.731828 15231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:36:39.731471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.129:35757 (local address 127.0.0.1:52510)
0809 09:36:39.731664 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0809 09:36:39.731738 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.129:35757: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104}
W20260809 09:36:39.732093 14511 catalog_manager.cc:4584] TS 9ea20a89428d477b81ce6458c5648c3d (127.20.125.129:35757): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e65c77582d2f4c46b845f83409f5562a: Network error: Client connection negotiation failed: client connection to 127.20.125.129:35757: connect: Connection refused (error 111)
I20260809 09:36:39.732548 14508 catalog_manager.cc:4841] TS f83294e89ecb4acebde75ecd689275bc (127.20.125.131:36735): tablet e65c77582d2f4c46b845f83409f5562a (table test-table [id=5e9450bec1e24220a6076df89e74c69b]) successfully deleted
I20260809 09:36:39.737449 20982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu with pid 15010
I20260809 09:36:39.746289 20982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu with pid 14486
2026-08-09T09:36:39Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (1056 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786267941944493-20982-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 09:36:39 dist-test-slave-7kzw krb5kdc[15281](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 09:36:39 dist-test-slave-7kzw krb5kdc[15281](info): set up 2 sockets
Aug 09 09:36:39 dist-test-slave-7kzw krb5kdc[15281](info): commencing operation
krb5kdc: starting...
W20260809 09:36:40.983628 20982 mini_kdc.cc:121] Time spent starting KDC: real 1.212s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786267941944493-20982-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786267941944493-20982-0/krb5kdc/test-user.keytab.
Aug 09 09:36:41 dist-test-slave-7kzw krb5kdc[15281](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268201, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T09:36:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T09:36:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.125.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.125.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.125.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.190.keytab.
Entry for principal kudu/127.20.125.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.190.keytab.
I20260809 09:36:41.114244 20982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
/tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786267941944493-20982-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786267941944493-20982-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786267941944493-20982-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786267941944493-20982-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.125.190:37253
--webserver_interface=127.20.125.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.190.keytab
--principal=kudu/127.20.125.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.125.148:36539
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.125.190:37253
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786267941944493-20982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786267941944493-20982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786267941944493-20982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 09:36:41.230705 15298 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 09:36:41.231034 15298 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 09:36:41.231119 15298 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 09:36:41.235272 15298 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 09:36:41.235360 15298 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 09:36:41.235399 15298 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 09:36:41.235458 15298 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 09:36:41.235510 15298 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 09:36:41.240986 15298 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.125.148:36539
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786267941944493-20982-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786267941944493-20982-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.125.190:37253
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.190.keytab
--principal=kudu/127.20.125.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.125.190:37253
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786267941944493-20982-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.125.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15298
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786267941944493-20982-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 08:38:55 UTC on ccycloud.quasar-njjleu.root.comops.site
build id 4575935
I20260809 09:36:41.242108 15298 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 09:36:41.243259 15298 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 09:36:41.252048 15306 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:36:41.252084 15312 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:36:41.252156 15307 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:36:41.252318 15298 server_base.cc:1034] running on GCE node
I20260809 09:36:41.253167 15298 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 09:36:41.254288 15298 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 09:36:41.255466 15298 hybrid_clock.cc:648] HybridClock initialized: now 1786268201255439 us; error 43 us; skew 500 ppm
Aug 09 09:36:41 dist-test-slave-7kzw krb5kdc[15281](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268201, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 09:36:41.261973 15298 init.cc:377] Logged in from keytab as kudu/127.20.125.190@KRBTEST.COM (short username kudu)
I20260809 09:36:41.262991 15298 webserver.cc:460] Webserver started at http://127.20.125.190:39373/ using document root <none> and password file <none>
I20260809 09:36:41.263586 15298 fs_manager.cc:359] Metadata directory not provided
I20260809 09:36:41.263650 15298 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 09:36:41.263901 15298 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 09:36:41.265965 15298 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786267941944493-20982-0/minicluster-data/master-0/data/instance:
uuid: "afdf2e5e322041729408591db46e6a8d"
format_stamp: "Formatted at 2026-08-09 09:36:41 on dist-test-slave-7kzw"
server_key: "fbc8a1b3bf0f5304d7fce71dabf4ef75"
server_key_iv: "db2310df8069984376a72651d3ad49a9"
server_key_version: "encryptionkey@0"
I20260809 09:36:41.266608 15298 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786267941944493-20982-0/minicluster-data/master-0/wal/instance:
uuid: "afdf2e5e322041729408591db46e6a8d"
format_stamp: "Formatted at 2026-08-09 09:36:41 on dist-test-slave-7kzw"
server_key: "fbc8a1b3bf0f5304d7fce71dabf4ef75"
server_key_iv: "db2310df8069984376a72651d3ad49a9"
server_key_version: "encryptionkey@0"
I20260809 09:36:41.270256 15298 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260809 09:36:41.272835 15327 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 09:36:41.274035 15298 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 09:36:41.274195 15298 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786267941944493-20982-0/minicluster-data/master-0/data,/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786267941944493-20982-0/minicluster-data/master-0/wal
uuid: "afdf2e5e322041729408591db46e6a8d"
format_stamp: "Formatted at 2026-08-09 09:36:41 on dist-test-slave-7kzw"
server_key: "fbc8a1b3bf0f5304d7fce71dabf4ef75"
server_key_iv: "db2310df8069984376a72651d3ad49a9"
server_key_version: "encryptionkey@0"
I20260809 09:36:41.274312 15298 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786267941944493-20982-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786267941944493-20982-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786267941944493-20982-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 09:36:41.297865 15298 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 09:36:41.301258 15298 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 09:36:41.301508 15298 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 09:36:41.309216 15298 rpc_server.cc:307] RPC server started. Bound to: 127.20.125.190:37253
I20260809 09:36:41.309240 15401 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.125.190:37253 every 8 connection(s)
I20260809 09:36:41.310225 15298 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786267941944493-20982-0/minicluster-data/master-0/data/info.pb
I20260809 09:36:41.311350 20982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu as pid 15298
I20260809 09:36:41.311496 20982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786267941944493-20982-0/minicluster-data/master-0/wal/instance
I20260809 09:36:41.311847 20982 external_mini_cluster.cc:1428] Setting key d1e28b999525792efdd6cd3781dec55f
I20260809 09:36:41.313819 15403 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 09:36:41.320879 15403 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P afdf2e5e322041729408591db46e6a8d: Bootstrap starting.
Aug 09 09:36:41 dist-test-slave-7kzw krb5kdc[15281](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268201, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.125.190@KRBTEST.COM
I20260809 09:36:41.323508 15403 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P afdf2e5e322041729408591db46e6a8d: Neither blocks nor log segments found. Creating new log.
I20260809 09:36:41.324501 15403 log.cc:824] T 00000000000000000000000000000000 P afdf2e5e322041729408591db46e6a8d: Log is configured to *not* fsync() on all Append() calls
I20260809 09:36:41.326804 15403 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P afdf2e5e322041729408591db46e6a8d: No bootstrap required, opened a new log
I20260809 09:36:41.328545 15407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:36:41.313940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34148 (local address 127.20.125.190:37253)
0809 09:36:41.314413 (+ 473us) server_negotiation.cc:206] Beginning negotiation
0809 09:36:41.314440 (+ 27us) server_negotiation.cc:399] Waiting for connection header
0809 09:36:41.314512 (+ 72us) server_negotiation.cc:407] Connection header received
0809 09:36:41.315382 (+ 870us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:36:41.315406 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:36:41.315815 (+ 409us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:36:41.316178 (+ 363us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:36:41.316960 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:36:41.318451 (+ 1491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:36:41.319415 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:36:41.320118 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:36:41.320286 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:36:41.323236 (+ 2950us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:36:41.323243 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:36:41.323259 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:36:41.323291 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:36:41.325231 (+ 1940us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:36:41.325722 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:36:41.325728 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:36:41.325734 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:36:41.325870 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:36:41.326204 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:36:41.326207 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:36:41.326211 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:36:41.326812 (+ 601us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:36:41.327061 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0809 09:36:41.327452 (+ 391us) server_negotiation.cc:299] Negotiation successful
0809 09:36:41.327791 (+ 339us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":314,"thread_start_us":144,"threads_started":1}
I20260809 09:36:41.330442 15403 raft_consensus.cc:348] T 00000000000000000000000000000000 P afdf2e5e322041729408591db46e6a8d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "afdf2e5e322041729408591db46e6a8d" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 37253 } }
I20260809 09:36:41.330761 15403 raft_consensus.cc:374] T 00000000000000000000000000000000 P afdf2e5e322041729408591db46e6a8d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 09:36:41.330822 15403 raft_consensus.cc:729] T 00000000000000000000000000000000 P afdf2e5e322041729408591db46e6a8d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: afdf2e5e322041729408591db46e6a8d, State: Initialized, Role: FOLLOWER
I20260809 09:36:41.331265 15403 consensus_queue.cc:260] T 00000000000000000000000000000000 P afdf2e5e322041729408591db46e6a8d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "afdf2e5e322041729408591db46e6a8d" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 37253 } }
I20260809 09:36:41.331389 15403 raft_consensus.cc:388] T 00000000000000000000000000000000 P afdf2e5e322041729408591db46e6a8d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 09:36:41.331468 15403 raft_consensus.cc:482] T 00000000000000000000000000000000 P afdf2e5e322041729408591db46e6a8d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 09:36:41.331590 15403 raft_consensus.cc:3037] T 00000000000000000000000000000000 P afdf2e5e322041729408591db46e6a8d [term 0 FOLLOWER]: Advancing to term 1
I20260809 09:36:41.332494 15403 raft_consensus.cc:504] T 00000000000000000000000000000000 P afdf2e5e322041729408591db46e6a8d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "afdf2e5e322041729408591db46e6a8d" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 37253 } }
I20260809 09:36:41.332831 15403 leader_election.cc:302] T 00000000000000000000000000000000 P afdf2e5e322041729408591db46e6a8d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: afdf2e5e322041729408591db46e6a8d; no voters:
I20260809 09:36:41.333133 15403 leader_election.cc:288] T 00000000000000000000000000000000 P afdf2e5e322041729408591db46e6a8d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 09:36:41.333338 15410 raft_consensus.cc:2781] T 00000000000000000000000000000000 P afdf2e5e322041729408591db46e6a8d [term 1 FOLLOWER]: Leader election won for term 1
I20260809 09:36:41.333666 15410 raft_consensus.cc:686] T 00000000000000000000000000000000 P afdf2e5e322041729408591db46e6a8d [term 1 LEADER]: Becoming Leader. State: Replica: afdf2e5e322041729408591db46e6a8d, State: Running, Role: LEADER
I20260809 09:36:41.334007 15410 consensus_queue.cc:237] T 00000000000000000000000000000000 P afdf2e5e322041729408591db46e6a8d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "afdf2e5e322041729408591db46e6a8d" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 37253 } }
I20260809 09:36:41.334388 15403 sys_catalog.cc:564] T 00000000000000000000000000000000 P afdf2e5e322041729408591db46e6a8d [sys.catalog]: configured and running, proceeding with master startup.
I20260809 09:36:41.335990 15411 sys_catalog.cc:455] T 00000000000000000000000000000000 P afdf2e5e322041729408591db46e6a8d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "afdf2e5e322041729408591db46e6a8d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "afdf2e5e322041729408591db46e6a8d" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 37253 } } }
I20260809 09:36:41.336163 15411 sys_catalog.cc:458] T 00000000000000000000000000000000 P afdf2e5e322041729408591db46e6a8d [sys.catalog]: This master's current role is: LEADER
I20260809 09:36:41.336551 15427 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 09:36:41.336024 15413 sys_catalog.cc:455] T 00000000000000000000000000000000 P afdf2e5e322041729408591db46e6a8d [sys.catalog]: SysCatalogTable state changed. Reason: New leader afdf2e5e322041729408591db46e6a8d. Latest consensus state: current_term: 1 leader_uuid: "afdf2e5e322041729408591db46e6a8d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "afdf2e5e322041729408591db46e6a8d" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 37253 } } }
I20260809 09:36:41.337277 15413 sys_catalog.cc:458] T 00000000000000000000000000000000 P afdf2e5e322041729408591db46e6a8d [sys.catalog]: This master's current role is: LEADER
I20260809 09:36:41.339821 15427 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 09:36:41.345521 15427 catalog_manager.cc:1347] Generated new cluster ID: 79898f9eb6b740c1990c8fc68451d347
I20260809 09:36:41.345613 15427 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 09:36:41.359965 15427 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 09:36:41.360956 15427 catalog_manager.cc:1504] Loading token signing keys...
I20260809 09:36:41.372175 15427 catalog_manager.cc:5868] T 00000000000000000000000000000000 P afdf2e5e322041729408591db46e6a8d: Generated new TSK 0
I20260809 09:36:41.372984 15427 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.20.125.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.125.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.125.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.129.keytab.
Entry for principal kudu/127.20.125.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.129.keytab.
I20260809 09:36:41.413686 20982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
/tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786267941944493-20982-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786267941944493-20982-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786267941944493-20982-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786267941944493-20982-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.125.129:0
--local_ip_for_outbound_sockets=127.20.125.129
--webserver_interface=127.20.125.129
--webserver_port=0
--tserver_master_addrs=127.20.125.190:37253
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.129.keytab
--principal=kudu/127.20.125.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.125.148:36539
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786267941944493-20982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786267941944493-20982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786267941944493-20982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 09:36:41.527878 15442 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 09:36:41.528223 15442 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 09:36:41.528272 15442 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 09:36:41.532584 15442 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 09:36:41.532667 15442 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 09:36:41.532728 15442 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.125.129
I20260809 09:36:41.538861 15442 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.125.148:36539
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786267941944493-20982-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786267941944493-20982-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.129.keytab
--principal=kudu/127.20.125.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.125.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786267941944493-20982-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.125.129
--webserver_port=0
--tserver_master_addrs=127.20.125.190:37253
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15442
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.125.129
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786267941944493-20982-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 08:38:55 UTC on ccycloud.quasar-njjleu.root.comops.site
build id 4575935
I20260809 09:36:41.540081 15442 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 09:36:41.541203 15442 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 09:36:41.551225 15451 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:36:41.551260 15450 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:36:41.551188 15453 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:36:41.551388 15442 server_base.cc:1034] running on GCE node
I20260809 09:36:41.551901 15442 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 09:36:41.552635 15442 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 09:36:41.553784 15442 hybrid_clock.cc:648] HybridClock initialized: now 1786268201553759 us; error 36 us; skew 500 ppm
Aug 09 09:36:41 dist-test-slave-7kzw krb5kdc[15281](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268201, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 09:36:41.556780 15442 init.cc:377] Logged in from keytab as kudu/127.20.125.129@KRBTEST.COM (short username kudu)
I20260809 09:36:41.557876 15442 webserver.cc:460] Webserver started at http://127.20.125.129:46087/ using document root <none> and password file <none>
I20260809 09:36:41.558568 15442 fs_manager.cc:359] Metadata directory not provided
I20260809 09:36:41.558650 15442 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 09:36:41.558892 15442 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 09:36:41.561142 15442 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786267941944493-20982-0/minicluster-data/ts-0/data/instance:
uuid: "3f409f7879d442e292cf8a17d88ba28e"
format_stamp: "Formatted at 2026-08-09 09:36:41 on dist-test-slave-7kzw"
server_key: "34876dbadf9e14a3e53f8438dd78f3ae"
server_key_iv: "11c62485964f091dddc599456f73d246"
server_key_version: "encryptionkey@0"
I20260809 09:36:41.561810 15442 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786267941944493-20982-0/minicluster-data/ts-0/wal/instance:
uuid: "3f409f7879d442e292cf8a17d88ba28e"
format_stamp: "Formatted at 2026-08-09 09:36:41 on dist-test-slave-7kzw"
server_key: "34876dbadf9e14a3e53f8438dd78f3ae"
server_key_iv: "11c62485964f091dddc599456f73d246"
server_key_version: "encryptionkey@0"
I20260809 09:36:41.566066 15442 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260809 09:36:41.568990 15464 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 09:36:41.570721 15442 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 09:36:41.570909 15442 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786267941944493-20982-0/minicluster-data/ts-0/data,/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786267941944493-20982-0/minicluster-data/ts-0/wal
uuid: "3f409f7879d442e292cf8a17d88ba28e"
format_stamp: "Formatted at 2026-08-09 09:36:41 on dist-test-slave-7kzw"
server_key: "34876dbadf9e14a3e53f8438dd78f3ae"
server_key_iv: "11c62485964f091dddc599456f73d246"
server_key_version: "encryptionkey@0"
I20260809 09:36:41.571051 15442 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786267941944493-20982-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786267941944493-20982-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786267941944493-20982-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 09:36:41.594197 15442 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 09:36:41.597671 15442 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 09:36:41.597908 15442 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 09:36:41.598572 15442 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 09:36:41.599443 15442 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 09:36:41.599530 15442 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 09:36:41.599627 15442 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 09:36:41.599678 15442 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 09:36:41.608654 15442 rpc_server.cc:307] RPC server started. Bound to: 127.20.125.129:45767
I20260809 09:36:41.608681 15645 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.125.129:45767 every 8 connection(s)
I20260809 09:36:41.609551 15442 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786267941944493-20982-0/minicluster-data/ts-0/data/info.pb
I20260809 09:36:41.611497 20982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu as pid 15442
I20260809 09:36:41.611651 20982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786267941944493-20982-0/minicluster-data/ts-0/wal/instance
I20260809 09:36:41.611974 20982 external_mini_cluster.cc:1428] Setting key 1ead4790f5b43e89cf15ae12f752d984
Aug 09 09:36:41 dist-test-slave-7kzw krb5kdc[15281](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268201, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.129@KRBTEST.COM for kudu/127.20.125.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.20.125.130@KRBTEST.COM; defaulting to no policy
I20260809 09:36:41.624236 15407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:36:41.611558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.129:38667 (local address 127.20.125.190:37253)
0809 09:36:41.611728 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0809 09:36:41.611733 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 09:36:41.612636 (+ 903us) server_negotiation.cc:407] Connection header received
0809 09:36:41.613995 (+ 1359us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:36:41.614000 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:36:41.614063 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:36:41.614155 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:36:41.615401 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:36:41.616018 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:36:41.617171 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:36:41.617442 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:36:41.617559 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:36:41.620450 (+ 2891us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:36:41.620453 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:36:41.620455 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:36:41.620478 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:36:41.622014 (+ 1536us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:36:41.622575 (+ 561us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:36:41.622579 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:36:41.622581 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:36:41.622665 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:36:41.623066 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:36:41.623069 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:36:41.623070 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:36:41.623248 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:36:41.623355 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0809 09:36:41.623858 (+ 503us) server_negotiation.cc:299] Negotiation successful
0809 09:36:41.624025 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.125.130@KRBTEST.COM" created.
I20260809 09:36:41.625120 15652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:36:41.611935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:37253 (local address 127.20.125.129:38667)
0809 09:36:41.612471 (+ 536us) negotiation.cc:107] Waiting for socket to connect
0809 09:36:41.612525 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0809 09:36:41.613777 (+ 1252us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:36:41.614316 (+ 539us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:36:41.614322 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:36:41.614771 (+ 449us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:36:41.615242 (+ 471us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:36:41.615256 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:36:41.616157 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:36:41.616161 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:36:41.617036 (+ 875us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:36:41.617046 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:36:41.617602 (+ 556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:36:41.617606 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:36:41.617778 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:36:41.618442 (+ 664us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 09:36:41.618458 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:36:41.620274 (+ 1816us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 09:36:41.622140 (+ 1866us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:36:41.622147 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:36:41.622161 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:36:41.622434 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:36:41.622779 (+ 345us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:36:41.622782 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:36:41.622784 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:36:41.622933 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:36:41.623354 (+ 421us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 09:36:41.623363 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 09:36:41.623614 (+ 251us) client_negotiation.cc:769] Sending connection context
0809 09:36:41.623799 (+ 185us) client_negotiation.cc:240] Negotiation successful
0809 09:36:41.624126 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":331,"thread_start_us":149,"threads_started":1}
I20260809 09:36:41.626461 15646 heartbeater.cc:347] Connected to a master server at 127.20.125.190:37253
I20260809 09:36:41.626780 15646 heartbeater.cc:464] Registering TS with master...
I20260809 09:36:41.627282 15646 heartbeater.cc:511] Master 127.20.125.190:37253 requested a full tablet report, sending...
I20260809 09:36:41.628822 15351 ts_manager.cc:194] Registered new tserver with Master: 3f409f7879d442e292cf8a17d88ba28e (127.20.125.129:45767)
I20260809 09:36:41.629940 15351 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.125.129@KRBTEST.COM'} at 127.20.125.129:38667
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.125.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.130.keytab.
Entry for principal kudu/127.20.125.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.130.keytab.
I20260809 09:36:41.639788 20982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
/tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786267941944493-20982-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786267941944493-20982-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786267941944493-20982-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786267941944493-20982-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.125.130:0
--local_ip_for_outbound_sockets=127.20.125.130
--webserver_interface=127.20.125.130
--webserver_port=0
--tserver_master_addrs=127.20.125.190:37253
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.130.keytab
--principal=kudu/127.20.125.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.125.148:36539
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786267941944493-20982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786267941944493-20982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786267941944493-20982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 09:36:41.756805 15656 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 09:36:41.757128 15656 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 09:36:41.757228 15656 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 09:36:41.761413 15656 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 09:36:41.761504 15656 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 09:36:41.761600 15656 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.125.130
I20260809 09:36:41.766892 15656 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.125.148:36539
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786267941944493-20982-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786267941944493-20982-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.130.keytab
--principal=kudu/127.20.125.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.125.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786267941944493-20982-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.125.130
--webserver_port=0
--tserver_master_addrs=127.20.125.190:37253
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15656
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.125.130
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786267941944493-20982-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 08:38:55 UTC on ccycloud.quasar-njjleu.root.comops.site
build id 4575935
I20260809 09:36:41.767962 15656 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 09:36:41.769222 15656 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 09:36:41.779939 15667 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:36:41.780041 15656 server_base.cc:1034] running on GCE node
W20260809 09:36:41.779932 15664 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:36:41.779949 15663 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:36:41.780515 15656 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 09:36:41.781162 15656 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 09:36:41.782336 15656 hybrid_clock.cc:648] HybridClock initialized: now 1786268201782322 us; error 26 us; skew 500 ppm
Aug 09 09:36:41 dist-test-slave-7kzw krb5kdc[15281](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268201, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 09:36:41.785140 15656 init.cc:377] Logged in from keytab as kudu/127.20.125.130@KRBTEST.COM (short username kudu)
I20260809 09:36:41.786026 15656 webserver.cc:460] Webserver started at http://127.20.125.130:33915/ using document root <none> and password file <none>
I20260809 09:36:41.786604 15656 fs_manager.cc:359] Metadata directory not provided
I20260809 09:36:41.786672 15656 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 09:36:41.786849 15656 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 09:36:41.788808 15656 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786267941944493-20982-0/minicluster-data/ts-1/data/instance:
uuid: "187fc846f7674216b88372283f27064a"
format_stamp: "Formatted at 2026-08-09 09:36:41 on dist-test-slave-7kzw"
server_key: "4d516f47104ddb477f982ad9575ff71b"
server_key_iv: "4695a89f305827106451abf8c5e6f941"
server_key_version: "encryptionkey@0"
I20260809 09:36:41.789311 15656 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786267941944493-20982-0/minicluster-data/ts-1/wal/instance:
uuid: "187fc846f7674216b88372283f27064a"
format_stamp: "Formatted at 2026-08-09 09:36:41 on dist-test-slave-7kzw"
server_key: "4d516f47104ddb477f982ad9575ff71b"
server_key_iv: "4695a89f305827106451abf8c5e6f941"
server_key_version: "encryptionkey@0"
I20260809 09:36:41.793040 15656 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260809 09:36:41.795532 15677 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 09:36:41.796906 15656 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260809 09:36:41.797005 15656 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786267941944493-20982-0/minicluster-data/ts-1/data,/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786267941944493-20982-0/minicluster-data/ts-1/wal
uuid: "187fc846f7674216b88372283f27064a"
format_stamp: "Formatted at 2026-08-09 09:36:41 on dist-test-slave-7kzw"
server_key: "4d516f47104ddb477f982ad9575ff71b"
server_key_iv: "4695a89f305827106451abf8c5e6f941"
server_key_version: "encryptionkey@0"
I20260809 09:36:41.797089 15656 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786267941944493-20982-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786267941944493-20982-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786267941944493-20982-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 09:36:41.826536 15656 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 09:36:41.829993 15656 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 09:36:41.830161 15656 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 09:36:41.830749 15656 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 09:36:41.831564 15656 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 09:36:41.831612 15656 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 09:36:41.831655 15656 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 09:36:41.831677 15656 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 09:36:41.841337 15656 rpc_server.cc:307] RPC server started. Bound to: 127.20.125.130:38727
I20260809 09:36:41.841369 15855 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.125.130:38727 every 8 connection(s)
I20260809 09:36:41.842221 15656 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786267941944493-20982-0/minicluster-data/ts-1/data/info.pb
I20260809 09:36:41.847414 20982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu as pid 15656
I20260809 09:36:41.847551 20982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786267941944493-20982-0/minicluster-data/ts-1/wal/instance
I20260809 09:36:41.847855 20982 external_mini_cluster.cc:1428] Setting key 677b456d3a67f16d55b200f37d75dd31
Aug 09 09:36:41 dist-test-slave-7kzw krb5kdc[15281](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268201, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.130@KRBTEST.COM for kudu/127.20.125.190@KRBTEST.COM
I20260809 09:36:41.855988 15407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:36:41.844021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.130:58595 (local address 127.20.125.190:37253)
0809 09:36:41.844184 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0809 09:36:41.844189 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 09:36:41.845015 (+ 826us) server_negotiation.cc:407] Connection header received
0809 09:36:41.846308 (+ 1293us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:36:41.846311 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:36:41.846359 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:36:41.846436 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:36:41.847767 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:36:41.848379 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:36:41.849534 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:36:41.849778 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:36:41.849890 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:36:41.852428 (+ 2538us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:36:41.852431 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:36:41.852434 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:36:41.852457 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:36:41.853813 (+ 1356us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:36:41.854333 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:36:41.854337 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:36:41.854339 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:36:41.854456 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:36:41.854838 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:36:41.854841 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:36:41.854842 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:36:41.855028 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:36:41.855136 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0809 09:36:41.855677 (+ 541us) server_negotiation.cc:299] Negotiation successful
0809 09:36:41.855814 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260809 09:36:41.856786 15859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:36:41.844403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:37253 (local address 127.20.125.130:58595)
0809 09:36:41.844861 (+ 458us) negotiation.cc:107] Waiting for socket to connect
0809 09:36:41.844907 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 09:36:41.846089 (+ 1182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:36:41.846668 (+ 579us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:36:41.846675 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:36:41.847058 (+ 383us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:36:41.847574 (+ 516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:36:41.847594 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:36:41.848513 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:36:41.848516 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:36:41.849404 (+ 888us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:36:41.849413 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:36:41.849932 (+ 519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:36:41.849936 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:36:41.850108 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:36:41.850752 (+ 644us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 09:36:41.850769 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:36:41.852253 (+ 1484us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 09:36:41.853933 (+ 1680us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:36:41.853939 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:36:41.853952 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:36:41.854225 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:36:41.854568 (+ 343us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:36:41.854571 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:36:41.854573 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:36:41.854728 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:36:41.855136 (+ 408us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 09:36:41.855144 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 09:36:41.855399 (+ 255us) client_negotiation.cc:769] Sending connection context
0809 09:36:41.855621 (+ 222us) client_negotiation.cc:240] Negotiation successful
0809 09:36:41.855957 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":267,"thread_start_us":108,"threads_started":1}
I20260809 09:36:41.858083 15856 heartbeater.cc:347] Connected to a master server at 127.20.125.190:37253
I20260809 09:36:41.858328 15856 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.20.125.131@KRBTEST.COM; defaulting to no policy
I20260809 09:36:41.858925 15856 heartbeater.cc:511] Master 127.20.125.190:37253 requested a full tablet report, sending...
I20260809 09:36:41.860006 15351 ts_manager.cc:194] Registered new tserver with Master: 187fc846f7674216b88372283f27064a (127.20.125.130:38727)
I20260809 09:36:41.860627 15351 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.125.130@KRBTEST.COM'} at 127.20.125.130:58595
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.125.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.125.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.131.keytab.
Entry for principal kudu/127.20.125.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.131.keytab.
I20260809 09:36:41.878013 20982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
/tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786267941944493-20982-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786267941944493-20982-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786267941944493-20982-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786267941944493-20982-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.125.131:0
--local_ip_for_outbound_sockets=127.20.125.131
--webserver_interface=127.20.125.131
--webserver_port=0
--tserver_master_addrs=127.20.125.190:37253
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.131.keytab
--principal=kudu/127.20.125.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.125.148:36539
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786267941944493-20982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786267941944493-20982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786267941944493-20982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 09:36:41.993364 15863 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 09:36:41.993646 15863 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 09:36:41.993721 15863 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 09:36:41.997789 15863 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 09:36:41.997884 15863 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 09:36:41.997973 15863 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.125.131
I20260809 09:36:42.003487 15863 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.125.148:36539
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786267941944493-20982-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786267941944493-20982-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.131.keytab
--principal=kudu/127.20.125.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.125.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786267941944493-20982-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.125.131
--webserver_port=0
--tserver_master_addrs=127.20.125.190:37253
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15863
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.125.131
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786267941944493-20982-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 08:38:55 UTC on ccycloud.quasar-njjleu.root.comops.site
build id 4575935
I20260809 09:36:42.004642 15863 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 09:36:42.005704 15863 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 09:36:42.015573 15874 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:36:42.015805 15863 server_base.cc:1034] running on GCE node
W20260809 09:36:42.016012 15875 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:36:42.016019 15878 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:36:42.016532 15863 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 09:36:42.017163 15863 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 09:36:42.018345 15863 hybrid_clock.cc:648] HybridClock initialized: now 1786268202018326 us; error 32 us; skew 500 ppm
Aug 09 09:36:42 dist-test-slave-7kzw krb5kdc[15281](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268202, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 09:36:42.021605 15863 init.cc:377] Logged in from keytab as kudu/127.20.125.131@KRBTEST.COM (short username kudu)
I20260809 09:36:42.022733 15863 webserver.cc:460] Webserver started at http://127.20.125.131:35383/ using document root <none> and password file <none>
I20260809 09:36:42.023383 15863 fs_manager.cc:359] Metadata directory not provided
I20260809 09:36:42.023455 15863 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 09:36:42.023716 15863 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 09:36:42.025995 15863 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786267941944493-20982-0/minicluster-data/ts-2/data/instance:
uuid: "fd74d537218d4dfa83b73820ef992eb0"
format_stamp: "Formatted at 2026-08-09 09:36:42 on dist-test-slave-7kzw"
server_key: "9d9dfee72ed88b64fdaefa6797f5fb05"
server_key_iv: "08a0e971177ac31fc807de95cfbc5a85"
server_key_version: "encryptionkey@0"
I20260809 09:36:42.026724 15863 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786267941944493-20982-0/minicluster-data/ts-2/wal/instance:
uuid: "fd74d537218d4dfa83b73820ef992eb0"
format_stamp: "Formatted at 2026-08-09 09:36:42 on dist-test-slave-7kzw"
server_key: "9d9dfee72ed88b64fdaefa6797f5fb05"
server_key_iv: "08a0e971177ac31fc807de95cfbc5a85"
server_key_version: "encryptionkey@0"
I20260809 09:36:42.030999 15863 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260809 09:36:42.033658 15887 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 09:36:42.035199 15863 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 09:36:42.035351 15863 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786267941944493-20982-0/minicluster-data/ts-2/data,/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786267941944493-20982-0/minicluster-data/ts-2/wal
uuid: "fd74d537218d4dfa83b73820ef992eb0"
format_stamp: "Formatted at 2026-08-09 09:36:42 on dist-test-slave-7kzw"
server_key: "9d9dfee72ed88b64fdaefa6797f5fb05"
server_key_iv: "08a0e971177ac31fc807de95cfbc5a85"
server_key_version: "encryptionkey@0"
I20260809 09:36:42.035483 15863 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786267941944493-20982-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786267941944493-20982-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786267941944493-20982-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 09:36:42.051764 15863 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 09:36:42.055084 15863 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 09:36:42.055316 15863 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 09:36:42.055953 15863 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 09:36:42.056859 15863 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 09:36:42.056937 15863 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 09:36:42.057018 15863 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 09:36:42.057080 15863 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 09:36:42.067309 15863 rpc_server.cc:307] RPC server started. Bound to: 127.20.125.131:40895
I20260809 09:36:42.067339 16083 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.125.131:40895 every 8 connection(s)
I20260809 09:36:42.068197 15863 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786267941944493-20982-0/minicluster-data/ts-2/data/info.pb
I20260809 09:36:42.075892 20982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu as pid 15863
I20260809 09:36:42.076045 20982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786267941944493-20982-0/minicluster-data/ts-2/wal/instance
I20260809 09:36:42.076416 20982 external_mini_cluster.cc:1428] Setting key b7b7d4cd04f2a14ed784d04dbddfd12f
Aug 09 09:36:42 dist-test-slave-7kzw krb5kdc[15281](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268202, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.131@KRBTEST.COM for kudu/127.20.125.190@KRBTEST.COM
I20260809 09:36:42.082787 15407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:36:42.070084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.131:54983 (local address 127.20.125.190:37253)
0809 09:36:42.070239 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0809 09:36:42.070243 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 09:36:42.071129 (+ 886us) server_negotiation.cc:407] Connection header received
0809 09:36:42.072530 (+ 1401us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:36:42.072533 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:36:42.072593 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:36:42.072677 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:36:42.073793 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:36:42.074331 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:36:42.075604 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:36:42.075888 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:36:42.076042 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:36:42.079297 (+ 3255us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:36:42.079300 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:36:42.079302 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:36:42.079325 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:36:42.080723 (+ 1398us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:36:42.081268 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:36:42.081271 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:36:42.081272 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:36:42.081348 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:36:42.081672 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:36:42.081674 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:36:42.081675 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:36:42.081862 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:36:42.081952 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0809 09:36:42.082532 (+ 580us) server_negotiation.cc:299] Negotiation successful
0809 09:36:42.082645 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 09:36:42.083638 16088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:36:42.070440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:37253 (local address 127.20.125.131:54983)
0809 09:36:42.070968 (+ 528us) negotiation.cc:107] Waiting for socket to connect
0809 09:36:42.071028 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0809 09:36:42.072318 (+ 1290us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:36:42.072838 (+ 520us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:36:42.072845 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:36:42.073185 (+ 340us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:36:42.073641 (+ 456us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:36:42.073653 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:36:42.074461 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:36:42.074499 (+ 38us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:36:42.075401 (+ 902us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:36:42.075411 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:36:42.076127 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:36:42.076132 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:36:42.076334 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:36:42.076979 (+ 645us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 09:36:42.077002 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:36:42.079146 (+ 2144us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 09:36:42.080845 (+ 1699us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:36:42.080854 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:36:42.080866 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:36:42.081165 (+ 299us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:36:42.081433 (+ 268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:36:42.081436 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:36:42.081437 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:36:42.081578 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:36:42.081976 (+ 398us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 09:36:42.081982 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 09:36:42.082236 (+ 254us) client_negotiation.cc:769] Sending connection context
0809 09:36:42.082404 (+ 168us) client_negotiation.cc:240] Negotiation successful
0809 09:36:42.082764 (+ 360us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":261,"thread_start_us":104,"threads_started":1}
I20260809 09:36:42.084818 16084 heartbeater.cc:347] Connected to a master server at 127.20.125.190:37253
I20260809 09:36:42.085096 16084 heartbeater.cc:464] Registering TS with master...
I20260809 09:36:42.085633 16084 heartbeater.cc:511] Master 127.20.125.190:37253 requested a full tablet report, sending...
I20260809 09:36:42.086982 15351 ts_manager.cc:194] Registered new tserver with Master: fd74d537218d4dfa83b73820ef992eb0 (127.20.125.131:40895)
I20260809 09:36:42.087603 15351 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.125.131@KRBTEST.COM'} at 127.20.125.131:54983
I20260809 09:36:42.090832 20982 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 09:36:42.100800 15407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:36:42.092824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34158 (local address 127.20.125.190:37253)
0809 09:36:42.092981 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0809 09:36:42.092986 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 09:36:42.093142 (+ 156us) server_negotiation.cc:407] Connection header received
0809 09:36:42.093299 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:36:42.093302 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:36:42.093347 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:36:42.093431 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:36:42.094190 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:36:42.094751 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:36:42.095608 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:36:42.095822 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:36:42.095929 (+ 107us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:36:42.097532 (+ 1603us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:36:42.097537 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:36:42.097539 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:36:42.097567 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:36:42.098897 (+ 1330us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:36:42.099411 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:36:42.099417 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:36:42.099424 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:36:42.099495 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:36:42.099841 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:36:42.099844 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:36:42.099846 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:36:42.100089 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:36:42.100263 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0809 09:36:42.100537 (+ 274us) server_negotiation.cc:299] Negotiation successful
0809 09:36:42.100631 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260809 09:36:42.103703 15351 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34158:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 09:36:42.106793 15351 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 09:36:42.122314 16100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:36:42.116593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.129:45767 (local address 127.0.0.1:56366)
0809 09:36:42.117436 (+ 843us) negotiation.cc:107] Waiting for socket to connect
0809 09:36:42.117448 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 09:36:42.117522 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:36:42.118149 (+ 627us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:36:42.118153 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:36:42.118186 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 09:36:42.118276 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:36:42.118280 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:36:42.119336 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:36:42.119341 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:36:42.120620 (+ 1279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:36:42.120630 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:36:42.121748 (+ 1118us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:36:42.121751 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:36:42.121870 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:36:42.121907 (+ 37us) client_negotiation.cc:769] Sending connection context
0809 09:36:42.122026 (+ 119us) client_negotiation.cc:240] Negotiation successful
0809 09:36:42.122148 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":783,"mutex_wait_us":2,"thread_start_us":94,"threads_started":1}
I20260809 09:36:42.122326 16108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:36:42.116785 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56366 (local address 127.20.125.129:45767)
0809 09:36:42.117585 (+ 800us) server_negotiation.cc:206] Beginning negotiation
0809 09:36:42.117602 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0809 09:36:42.117624 (+ 22us) server_negotiation.cc:407] Connection header received
0809 09:36:42.117697 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:36:42.117704 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:36:42.117875 (+ 171us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:36:42.118016 (+ 141us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 09:36:42.118387 (+ 371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:36:42.119203 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:36:42.120748 (+ 1545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:36:42.121602 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:36:42.121751 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:36:42.121909 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0809 09:36:42.122068 (+ 159us) server_negotiation.cc:299] Negotiation successful
0809 09:36:42.122164 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":701,"thread_start_us":122,"threads_started":1}
I20260809 09:36:42.122661 16107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:36:42.116776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43368 (local address 127.20.125.130:38727)
0809 09:36:42.117506 (+ 730us) server_negotiation.cc:206] Beginning negotiation
0809 09:36:42.117518 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 09:36:42.117543 (+ 25us) server_negotiation.cc:407] Connection header received
0809 09:36:42.117623 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:36:42.117628 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:36:42.117825 (+ 197us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:36:42.117982 (+ 157us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 09:36:42.118187 (+ 205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:36:42.119352 (+ 1165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:36:42.121079 (+ 1727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:36:42.121632 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:36:42.121779 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:36:42.121910 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0809 09:36:42.122387 (+ 477us) server_negotiation.cc:299] Negotiation successful
0809 09:36:42.122535 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":620,"thread_start_us":102,"threads_started":1}
I20260809 09:36:42.122809 16102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:36:42.115942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.131:40895 (local address 127.0.0.1:50632)
0809 09:36:42.117009 (+ 1067us) negotiation.cc:107] Waiting for socket to connect
0809 09:36:42.117062 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0809 09:36:42.117224 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:36:42.117784 (+ 560us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:36:42.117790 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:36:42.117812 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 09:36:42.117967 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:36:42.117975 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:36:42.119519 (+ 1544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:36:42.119522 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:36:42.120825 (+ 1303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:36:42.120836 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:36:42.121904 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:36:42.121907 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:36:42.122029 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:36:42.122276 (+ 247us) client_negotiation.cc:769] Sending connection context
0809 09:36:42.122353 (+ 77us) client_negotiation.cc:240] Negotiation successful
0809 09:36:42.122658 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":880,"mutex_wait_us":273,"thread_start_us":88,"threads_started":1}
I20260809 09:36:42.122627 16105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:36:42.115978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.130:38727 (local address 127.0.0.1:43368)
0809 09:36:42.117144 (+ 1166us) negotiation.cc:107] Waiting for socket to connect
0809 09:36:42.117156 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 09:36:42.117223 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:36:42.117960 (+ 737us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:36:42.117965 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:36:42.117988 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 09:36:42.118085 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:36:42.118092 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:36:42.119757 (+ 1665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:36:42.119761 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:36:42.120950 (+ 1189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:36:42.120960 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:36:42.121978 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:36:42.121984 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:36:42.122121 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:36:42.122152 (+ 31us) client_negotiation.cc:769] Sending connection context
0809 09:36:42.122224 (+ 72us) client_negotiation.cc:240] Negotiation successful
0809 09:36:42.122281 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1094,"mutex_wait_us":60,"thread_start_us":112,"threads_started":1}
I20260809 09:36:42.123327 16104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:36:42.116073 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50632 (local address 127.20.125.131:40895)
0809 09:36:42.116474 (+ 401us) server_negotiation.cc:206] Beginning negotiation
0809 09:36:42.116485 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 09:36:42.117170 (+ 685us) server_negotiation.cc:407] Connection header received
0809 09:36:42.117389 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:36:42.117396 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:36:42.117603 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:36:42.117779 (+ 176us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 09:36:42.118103 (+ 324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:36:42.119357 (+ 1254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:36:42.120970 (+ 1613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:36:42.121767 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:36:42.122094 (+ 327us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:36:42.122214 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0809 09:36:42.123089 (+ 875us) server_negotiation.cc:299] Negotiation successful
0809 09:36:42.123197 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":292,"thread_start_us":179,"threads_started":1}
I20260809 09:36:42.125804 15756 tablet_service.cc:1467] Processing CreateTablet for tablet 8001930a232d4925bc5b11b4e3a390c3 (DEFAULT_TABLE table=test-table [id=51eed9cc72f846a8b8e58c96d067870b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 09:36:42.125856 15545 tablet_service.cc:1467] Processing CreateTablet for tablet 8001930a232d4925bc5b11b4e3a390c3 (DEFAULT_TABLE table=test-table [id=51eed9cc72f846a8b8e58c96d067870b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 09:36:42.125856 15969 tablet_service.cc:1467] Processing CreateTablet for tablet 8001930a232d4925bc5b11b4e3a390c3 (DEFAULT_TABLE table=test-table [id=51eed9cc72f846a8b8e58c96d067870b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 09:36:42.126909 15969 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8001930a232d4925bc5b11b4e3a390c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 09:36:42.126909 15545 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8001930a232d4925bc5b11b4e3a390c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 09:36:42.126915 15756 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8001930a232d4925bc5b11b4e3a390c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 09:36:42.133733 16110 tablet_bootstrap.cc:492] T 8001930a232d4925bc5b11b4e3a390c3 P 187fc846f7674216b88372283f27064a: Bootstrap starting.
I20260809 09:36:42.134791 16111 tablet_bootstrap.cc:492] T 8001930a232d4925bc5b11b4e3a390c3 P 3f409f7879d442e292cf8a17d88ba28e: Bootstrap starting.
I20260809 09:36:42.136730 16111 tablet_bootstrap.cc:654] T 8001930a232d4925bc5b11b4e3a390c3 P 3f409f7879d442e292cf8a17d88ba28e: Neither blocks nor log segments found. Creating new log.
I20260809 09:36:42.136730 16110 tablet_bootstrap.cc:654] T 8001930a232d4925bc5b11b4e3a390c3 P 187fc846f7674216b88372283f27064a: Neither blocks nor log segments found. Creating new log.
I20260809 09:36:42.137465 16111 log.cc:824] T 8001930a232d4925bc5b11b4e3a390c3 P 3f409f7879d442e292cf8a17d88ba28e: Log is configured to *not* fsync() on all Append() calls
I20260809 09:36:42.137806 16110 log.cc:824] T 8001930a232d4925bc5b11b4e3a390c3 P 187fc846f7674216b88372283f27064a: Log is configured to *not* fsync() on all Append() calls
I20260809 09:36:42.138163 16113 tablet_bootstrap.cc:492] T 8001930a232d4925bc5b11b4e3a390c3 P fd74d537218d4dfa83b73820ef992eb0: Bootstrap starting.
I20260809 09:36:42.139858 16111 tablet_bootstrap.cc:492] T 8001930a232d4925bc5b11b4e3a390c3 P 3f409f7879d442e292cf8a17d88ba28e: No bootstrap required, opened a new log
I20260809 09:36:42.139859 16110 tablet_bootstrap.cc:492] T 8001930a232d4925bc5b11b4e3a390c3 P 187fc846f7674216b88372283f27064a: No bootstrap required, opened a new log
I20260809 09:36:42.140064 16111 ts_tablet_manager.cc:1397] T 8001930a232d4925bc5b11b4e3a390c3 P 3f409f7879d442e292cf8a17d88ba28e: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260809 09:36:42.140064 16110 ts_tablet_manager.cc:1397] T 8001930a232d4925bc5b11b4e3a390c3 P 187fc846f7674216b88372283f27064a: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260809 09:36:42.140372 16113 tablet_bootstrap.cc:654] T 8001930a232d4925bc5b11b4e3a390c3 P fd74d537218d4dfa83b73820ef992eb0: Neither blocks nor log segments found. Creating new log.
I20260809 09:36:42.141642 16113 log.cc:824] T 8001930a232d4925bc5b11b4e3a390c3 P fd74d537218d4dfa83b73820ef992eb0: Log is configured to *not* fsync() on all Append() calls
I20260809 09:36:42.143797 16113 tablet_bootstrap.cc:492] T 8001930a232d4925bc5b11b4e3a390c3 P fd74d537218d4dfa83b73820ef992eb0: No bootstrap required, opened a new log
I20260809 09:36:42.143994 16113 ts_tablet_manager.cc:1397] T 8001930a232d4925bc5b11b4e3a390c3 P fd74d537218d4dfa83b73820ef992eb0: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260809 09:36:42.144142 16111 raft_consensus.cc:348] T 8001930a232d4925bc5b11b4e3a390c3 P 3f409f7879d442e292cf8a17d88ba28e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd74d537218d4dfa83b73820ef992eb0" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 40895 } } peers { permanent_uuid: "187fc846f7674216b88372283f27064a" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 38727 } } peers { permanent_uuid: "3f409f7879d442e292cf8a17d88ba28e" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 45767 } }
I20260809 09:36:42.144131 16110 raft_consensus.cc:348] T 8001930a232d4925bc5b11b4e3a390c3 P 187fc846f7674216b88372283f27064a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd74d537218d4dfa83b73820ef992eb0" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 40895 } } peers { permanent_uuid: "187fc846f7674216b88372283f27064a" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 38727 } } peers { permanent_uuid: "3f409f7879d442e292cf8a17d88ba28e" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 45767 } }
I20260809 09:36:42.144404 16110 raft_consensus.cc:374] T 8001930a232d4925bc5b11b4e3a390c3 P 187fc846f7674216b88372283f27064a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 09:36:42.144405 16111 raft_consensus.cc:374] T 8001930a232d4925bc5b11b4e3a390c3 P 3f409f7879d442e292cf8a17d88ba28e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 09:36:42.144469 16111 raft_consensus.cc:729] T 8001930a232d4925bc5b11b4e3a390c3 P 3f409f7879d442e292cf8a17d88ba28e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f409f7879d442e292cf8a17d88ba28e, State: Initialized, Role: FOLLOWER
I20260809 09:36:42.144500 16110 raft_consensus.cc:729] T 8001930a232d4925bc5b11b4e3a390c3 P 187fc846f7674216b88372283f27064a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 187fc846f7674216b88372283f27064a, State: Initialized, Role: FOLLOWER
I20260809 09:36:42.144948 16111 consensus_queue.cc:260] T 8001930a232d4925bc5b11b4e3a390c3 P 3f409f7879d442e292cf8a17d88ba28e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd74d537218d4dfa83b73820ef992eb0" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 40895 } } peers { permanent_uuid: "187fc846f7674216b88372283f27064a" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 38727 } } peers { permanent_uuid: "3f409f7879d442e292cf8a17d88ba28e" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 45767 } }
I20260809 09:36:42.144969 16110 consensus_queue.cc:260] T 8001930a232d4925bc5b11b4e3a390c3 P 187fc846f7674216b88372283f27064a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd74d537218d4dfa83b73820ef992eb0" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 40895 } } peers { permanent_uuid: "187fc846f7674216b88372283f27064a" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 38727 } } peers { permanent_uuid: "3f409f7879d442e292cf8a17d88ba28e" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 45767 } }
I20260809 09:36:42.145668 15856 heartbeater.cc:503] Master 127.20.125.190:37253 was elected leader, sending a full tablet report...
I20260809 09:36:42.145968 16111 ts_tablet_manager.cc:1428] T 8001930a232d4925bc5b11b4e3a390c3 P 3f409f7879d442e292cf8a17d88ba28e: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260809 09:36:42.145970 16110 ts_tablet_manager.cc:1428] T 8001930a232d4925bc5b11b4e3a390c3 P 187fc846f7674216b88372283f27064a: Time spent starting tablet: real 0.006s user 0.000s sys 0.007s
I20260809 09:36:42.146323 15646 heartbeater.cc:503] Master 127.20.125.190:37253 was elected leader, sending a full tablet report...
I20260809 09:36:42.148797 16113 raft_consensus.cc:348] T 8001930a232d4925bc5b11b4e3a390c3 P fd74d537218d4dfa83b73820ef992eb0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd74d537218d4dfa83b73820ef992eb0" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 40895 } } peers { permanent_uuid: "187fc846f7674216b88372283f27064a" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 38727 } } peers { permanent_uuid: "3f409f7879d442e292cf8a17d88ba28e" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 45767 } }
I20260809 09:36:42.149019 16113 raft_consensus.cc:374] T 8001930a232d4925bc5b11b4e3a390c3 P fd74d537218d4dfa83b73820ef992eb0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 09:36:42.149123 16113 raft_consensus.cc:729] T 8001930a232d4925bc5b11b4e3a390c3 P fd74d537218d4dfa83b73820ef992eb0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd74d537218d4dfa83b73820ef992eb0, State: Initialized, Role: FOLLOWER
I20260809 09:36:42.149545 16113 consensus_queue.cc:260] T 8001930a232d4925bc5b11b4e3a390c3 P fd74d537218d4dfa83b73820ef992eb0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd74d537218d4dfa83b73820ef992eb0" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 40895 } } peers { permanent_uuid: "187fc846f7674216b88372283f27064a" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 38727 } } peers { permanent_uuid: "3f409f7879d442e292cf8a17d88ba28e" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 45767 } }
I20260809 09:36:42.150439 16113 ts_tablet_manager.cc:1428] T 8001930a232d4925bc5b11b4e3a390c3 P fd74d537218d4dfa83b73820ef992eb0: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260809 09:36:42.150447 16084 heartbeater.cc:503] Master 127.20.125.190:37253 was elected leader, sending a full tablet report...
I20260809 09:36:42.218046 16121 raft_consensus.cc:482] T 8001930a232d4925bc5b11b4e3a390c3 P fd74d537218d4dfa83b73820ef992eb0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 09:36:42.218258 16121 raft_consensus.cc:504] T 8001930a232d4925bc5b11b4e3a390c3 P fd74d537218d4dfa83b73820ef992eb0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd74d537218d4dfa83b73820ef992eb0" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 40895 } } peers { permanent_uuid: "187fc846f7674216b88372283f27064a" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 38727 } } peers { permanent_uuid: "3f409f7879d442e292cf8a17d88ba28e" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 45767 } }
I20260809 09:36:42.219493 16121 leader_election.cc:288] T 8001930a232d4925bc5b11b4e3a390c3 P fd74d537218d4dfa83b73820ef992eb0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 187fc846f7674216b88372283f27064a (127.20.125.130:38727), 3f409f7879d442e292cf8a17d88ba28e (127.20.125.129:45767)
I20260809 09:36:42.224676 16107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:36:42.219864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.131:53619 (local address 127.20.125.130:38727)
0809 09:36:42.220009 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0809 09:36:42.220015 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 09:36:42.220034 (+ 19us) server_negotiation.cc:407] Connection header received
0809 09:36:42.220138 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:36:42.220143 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:36:42.220199 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:36:42.220294 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 09:36:42.220659 (+ 365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:36:42.221454 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:36:42.223196 (+ 1742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:36:42.223800 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:36:42.223938 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:36:42.223992 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0809 09:36:42.224473 (+ 481us) server_negotiation.cc:299] Negotiation successful
0809 09:36:42.224542 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260809 09:36:42.224707 16108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:36:42.219913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.131:40447 (local address 127.20.125.129:45767)
0809 09:36:42.220059 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0809 09:36:42.220066 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 09:36:42.220449 (+ 383us) server_negotiation.cc:407] Connection header received
0809 09:36:42.220544 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:36:42.220546 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:36:42.220617 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:36:42.220733 (+ 116us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 09:36:42.220981 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:36:42.221663 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:36:42.223227 (+ 1564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:36:42.223843 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:36:42.223947 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:36:42.223992 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0809 09:36:42.224473 (+ 481us) server_negotiation.cc:299] Negotiation successful
0809 09:36:42.224542 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 09:36:42.224735 16088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:36:42.219732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.130:38727 (local address 127.20.125.131:53619)
0809 09:36:42.219902 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0809 09:36:42.219921 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 09:36:42.220026 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:36:42.220324 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:36:42.220327 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:36:42.220351 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 09:36:42.220510 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:36:42.220529 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:36:42.221608 (+ 1079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:36:42.221611 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:36:42.223017 (+ 1406us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:36:42.223031 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:36:42.224072 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:36:42.224076 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:36:42.224283 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:36:42.224347 (+ 64us) client_negotiation.cc:769] Sending connection context
0809 09:36:42.224441 (+ 94us) client_negotiation.cc:240] Negotiation successful
0809 09:36:42.224560 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":65,"mutex_wait_us":150}
I20260809 09:36:42.224745 16123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:36:42.219732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.129:45767 (local address 127.20.125.131:40447)
0809 09:36:42.220337 (+ 605us) negotiation.cc:107] Waiting for socket to connect
0809 09:36:42.220351 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 09:36:42.220455 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:36:42.220737 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:36:42.220740 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:36:42.220758 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 09:36:42.220860 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:36:42.220865 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:36:42.221779 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:36:42.221783 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:36:42.223101 (+ 1318us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:36:42.223114 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:36:42.224072 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:36:42.224076 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:36:42.224280 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:36:42.224355 (+ 75us) client_negotiation.cc:769] Sending connection context
0809 09:36:42.224441 (+ 86us) client_negotiation.cc:240] Negotiation successful
0809 09:36:42.224560 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":535,"mutex_wait_us":334,"thread_start_us":158,"threads_started":1}
I20260809 09:36:42.225593 15581 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8001930a232d4925bc5b11b4e3a390c3" candidate_uuid: "fd74d537218d4dfa83b73820ef992eb0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f409f7879d442e292cf8a17d88ba28e" is_pre_election: true
I20260809 09:36:42.225709 15783 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8001930a232d4925bc5b11b4e3a390c3" candidate_uuid: "fd74d537218d4dfa83b73820ef992eb0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "187fc846f7674216b88372283f27064a" is_pre_election: true
I20260809 09:36:42.225939 15581 raft_consensus.cc:2454] T 8001930a232d4925bc5b11b4e3a390c3 P 3f409f7879d442e292cf8a17d88ba28e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fd74d537218d4dfa83b73820ef992eb0 in term 0.
I20260809 09:36:42.226066 15783 raft_consensus.cc:2454] T 8001930a232d4925bc5b11b4e3a390c3 P 187fc846f7674216b88372283f27064a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fd74d537218d4dfa83b73820ef992eb0 in term 0.
I20260809 09:36:42.226519 15888 leader_election.cc:302] T 8001930a232d4925bc5b11b4e3a390c3 P fd74d537218d4dfa83b73820ef992eb0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f409f7879d442e292cf8a17d88ba28e, fd74d537218d4dfa83b73820ef992eb0; no voters:
I20260809 09:36:42.226843 16121 raft_consensus.cc:2781] T 8001930a232d4925bc5b11b4e3a390c3 P fd74d537218d4dfa83b73820ef992eb0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 09:36:42.226938 16121 raft_consensus.cc:482] T 8001930a232d4925bc5b11b4e3a390c3 P fd74d537218d4dfa83b73820ef992eb0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 09:36:42.226997 16121 raft_consensus.cc:3037] T 8001930a232d4925bc5b11b4e3a390c3 P fd74d537218d4dfa83b73820ef992eb0 [term 0 FOLLOWER]: Advancing to term 1
I20260809 09:36:42.228133 16121 raft_consensus.cc:504] T 8001930a232d4925bc5b11b4e3a390c3 P fd74d537218d4dfa83b73820ef992eb0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd74d537218d4dfa83b73820ef992eb0" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 40895 } } peers { permanent_uuid: "187fc846f7674216b88372283f27064a" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 38727 } } peers { permanent_uuid: "3f409f7879d442e292cf8a17d88ba28e" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 45767 } }
I20260809 09:36:42.228565 16121 leader_election.cc:288] T 8001930a232d4925bc5b11b4e3a390c3 P fd74d537218d4dfa83b73820ef992eb0 [CANDIDATE]: Term 1 election: Requested vote from peers 187fc846f7674216b88372283f27064a (127.20.125.130:38727), 3f409f7879d442e292cf8a17d88ba28e (127.20.125.129:45767)
I20260809 09:36:42.228933 15783 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8001930a232d4925bc5b11b4e3a390c3" candidate_uuid: "fd74d537218d4dfa83b73820ef992eb0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "187fc846f7674216b88372283f27064a"
I20260809 09:36:42.229010 15581 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8001930a232d4925bc5b11b4e3a390c3" candidate_uuid: "fd74d537218d4dfa83b73820ef992eb0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f409f7879d442e292cf8a17d88ba28e"
I20260809 09:36:42.229074 15783 raft_consensus.cc:3037] T 8001930a232d4925bc5b11b4e3a390c3 P 187fc846f7674216b88372283f27064a [term 0 FOLLOWER]: Advancing to term 1
I20260809 09:36:42.229161 15581 raft_consensus.cc:3037] T 8001930a232d4925bc5b11b4e3a390c3 P 3f409f7879d442e292cf8a17d88ba28e [term 0 FOLLOWER]: Advancing to term 1
I20260809 09:36:42.230232 15783 raft_consensus.cc:2454] T 8001930a232d4925bc5b11b4e3a390c3 P 187fc846f7674216b88372283f27064a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fd74d537218d4dfa83b73820ef992eb0 in term 1.
I20260809 09:36:42.230244 15581 raft_consensus.cc:2454] T 8001930a232d4925bc5b11b4e3a390c3 P 3f409f7879d442e292cf8a17d88ba28e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fd74d537218d4dfa83b73820ef992eb0 in term 1.
I20260809 09:36:42.230574 15888 leader_election.cc:302] T 8001930a232d4925bc5b11b4e3a390c3 P fd74d537218d4dfa83b73820ef992eb0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f409f7879d442e292cf8a17d88ba28e, fd74d537218d4dfa83b73820ef992eb0; no voters:
I20260809 09:36:42.230767 16121 raft_consensus.cc:2781] T 8001930a232d4925bc5b11b4e3a390c3 P fd74d537218d4dfa83b73820ef992eb0 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 09:36:42.231051 16121 raft_consensus.cc:686] T 8001930a232d4925bc5b11b4e3a390c3 P fd74d537218d4dfa83b73820ef992eb0 [term 1 LEADER]: Becoming Leader. State: Replica: fd74d537218d4dfa83b73820ef992eb0, State: Running, Role: LEADER
I20260809 09:36:42.231379 16121 consensus_queue.cc:237] T 8001930a232d4925bc5b11b4e3a390c3 P fd74d537218d4dfa83b73820ef992eb0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd74d537218d4dfa83b73820ef992eb0" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 40895 } } peers { permanent_uuid: "187fc846f7674216b88372283f27064a" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 38727 } } peers { permanent_uuid: "3f409f7879d442e292cf8a17d88ba28e" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 45767 } }
I20260809 09:36:42.234586 15351 catalog_manager.cc:5495] T 8001930a232d4925bc5b11b4e3a390c3 P fd74d537218d4dfa83b73820ef992eb0 reported cstate change: term changed from 0 to 1, leader changed from <none> to fd74d537218d4dfa83b73820ef992eb0 (127.20.125.131). New cstate: current_term: 1 leader_uuid: "fd74d537218d4dfa83b73820ef992eb0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd74d537218d4dfa83b73820ef992eb0" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 40895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "187fc846f7674216b88372283f27064a" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 38727 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3f409f7879d442e292cf8a17d88ba28e" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 45767 } health_report { overall_health: UNKNOWN } } }
I20260809 09:36:42.247908 16104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:36:42.244163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50644 (local address 127.20.125.131:40895)
0809 09:36:42.244298 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0809 09:36:42.244301 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 09:36:42.244329 (+ 28us) server_negotiation.cc:407] Connection header received
0809 09:36:42.244512 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:36:42.244515 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:36:42.244568 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:36:42.244715 (+ 147us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 09:36:42.244976 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:36:42.245481 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:36:42.246297 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:36:42.246570 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:36:42.246663 (+ 93us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:36:42.247015 (+ 352us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 09:36:42.247404 (+ 389us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 09:36:42.247518 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0809 09:36:42.247703 (+ 185us) server_negotiation.cc:299] Negotiation successful
0809 09:36:42.247763 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260809 09:36:42.253633 15581 raft_consensus.cc:1261] T 8001930a232d4925bc5b11b4e3a390c3 P 3f409f7879d442e292cf8a17d88ba28e [term 1 FOLLOWER]: Refusing update from remote peer fd74d537218d4dfa83b73820ef992eb0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 09:36:42.253633 15783 raft_consensus.cc:1261] T 8001930a232d4925bc5b11b4e3a390c3 P 187fc846f7674216b88372283f27064a [term 1 FOLLOWER]: Refusing update from remote peer fd74d537218d4dfa83b73820ef992eb0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 09:36:42.254343 16124 consensus_queue.cc:1035] T 8001930a232d4925bc5b11b4e3a390c3 P fd74d537218d4dfa83b73820ef992eb0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f409f7879d442e292cf8a17d88ba28e" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 45767 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 09:36:42.254549 16121 consensus_queue.cc:1035] T 8001930a232d4925bc5b11b4e3a390c3 P fd74d537218d4dfa83b73820ef992eb0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "187fc846f7674216b88372283f27064a" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 38727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 09:36:42.263748 16130 mvcc.cc:205] Tried to move back new op lower bound from 7316554556423925760 to 7316554556342247424. Current Snapshot: MvccSnapshot[applied={T|T < 7316554556423925760}]
I20260809 09:36:42.263931 16131 mvcc.cc:205] Tried to move back new op lower bound from 7316554556423925760 to 7316554556342247424. Current Snapshot: MvccSnapshot[applied={T|T < 7316554556423925760}]
I20260809 09:36:42.272094 15348 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34158:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 09:36:42.272246 15348 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34158:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 09:36:42.274534 15348 catalog_manager.cc:5782] T 00000000000000000000000000000000 P afdf2e5e322041729408591db46e6a8d: Sending DeleteTablet for 3 replicas of tablet 8001930a232d4925bc5b11b4e3a390c3
I20260809 09:36:42.275319 15756 tablet_service.cc:1514] Processing DeleteTablet for tablet 8001930a232d4925bc5b11b4e3a390c3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 09:36:42 UTC) from {username='kudu', principal='kudu/127.20.125.190@KRBTEST.COM'} at 127.0.0.1:43368
I20260809 09:36:42.275333 15969 tablet_service.cc:1514] Processing DeleteTablet for tablet 8001930a232d4925bc5b11b4e3a390c3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 09:36:42 UTC) from {username='kudu', principal='kudu/127.20.125.190@KRBTEST.COM'} at 127.0.0.1:50632
I20260809 09:36:42.275693 15545 tablet_service.cc:1514] Processing DeleteTablet for tablet 8001930a232d4925bc5b11b4e3a390c3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 09:36:42 UTC) from {username='kudu', principal='kudu/127.20.125.190@KRBTEST.COM'} at 127.0.0.1:56366
I20260809 09:36:42.276155 16140 tablet_replica.cc:330] T 8001930a232d4925bc5b11b4e3a390c3 P 3f409f7879d442e292cf8a17d88ba28e: stopping tablet replica
I20260809 09:36:42.276321 16138 tablet_replica.cc:330] T 8001930a232d4925bc5b11b4e3a390c3 P 187fc846f7674216b88372283f27064a: stopping tablet replica
I20260809 09:36:42.276501 16140 raft_consensus.cc:2229] T 8001930a232d4925bc5b11b4e3a390c3 P 3f409f7879d442e292cf8a17d88ba28e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 09:36:42.276588 16138 raft_consensus.cc:2229] T 8001930a232d4925bc5b11b4e3a390c3 P 187fc846f7674216b88372283f27064a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 09:36:42.276870 20982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu with pid 15442
I20260809 09:36:42.276889 16140 raft_consensus.cc:2258] T 8001930a232d4925bc5b11b4e3a390c3 P 3f409f7879d442e292cf8a17d88ba28e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 09:36:42.276889 16138 raft_consensus.cc:2258] T 8001930a232d4925bc5b11b4e3a390c3 P 187fc846f7674216b88372283f27064a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 09:36:42.281085 16138 ts_tablet_manager.cc:1905] T 8001930a232d4925bc5b11b4e3a390c3 P 187fc846f7674216b88372283f27064a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 09:36:42.282714 16139 tablet_replica.cc:330] T 8001930a232d4925bc5b11b4e3a390c3 P fd74d537218d4dfa83b73820ef992eb0: stopping tablet replica
I20260809 09:36:42.283025 16139 raft_consensus.cc:2229] T 8001930a232d4925bc5b11b4e3a390c3 P fd74d537218d4dfa83b73820ef992eb0 [term 1 LEADER]: Raft consensus shutting down.
I20260809 09:36:42.283473 16139 raft_consensus.cc:2258] T 8001930a232d4925bc5b11b4e3a390c3 P fd74d537218d4dfa83b73820ef992eb0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 09:36:42.283521 16138 ts_tablet_manager.cc:1918] T 8001930a232d4925bc5b11b4e3a390c3 P 187fc846f7674216b88372283f27064a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 09:36:42.283607 16138 log.cc:1196] T 8001930a232d4925bc5b11b4e3a390c3 P 187fc846f7674216b88372283f27064a: Deleting WAL directory at /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786267941944493-20982-0/minicluster-data/ts-1/wal/wals/8001930a232d4925bc5b11b4e3a390c3
I20260809 09:36:42.283963 16138 ts_tablet_manager.cc:1939] T 8001930a232d4925bc5b11b4e3a390c3 P 187fc846f7674216b88372283f27064a: Deleting consensus metadata
I20260809 09:36:42.285743 16139 ts_tablet_manager.cc:1905] T 8001930a232d4925bc5b11b4e3a390c3 P fd74d537218d4dfa83b73820ef992eb0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 09:36:42.287216 15334 catalog_manager.cc:4841] TS 187fc846f7674216b88372283f27064a (127.20.125.130:38727): tablet 8001930a232d4925bc5b11b4e3a390c3 (table test-table [id=51eed9cc72f846a8b8e58c96d067870b]) successfully deleted
I20260809 09:36:42.288848 16139 ts_tablet_manager.cc:1918] T 8001930a232d4925bc5b11b4e3a390c3 P fd74d537218d4dfa83b73820ef992eb0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 09:36:42.288960 16139 log.cc:1196] T 8001930a232d4925bc5b11b4e3a390c3 P fd74d537218d4dfa83b73820ef992eb0: Deleting WAL directory at /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786267941944493-20982-0/minicluster-data/ts-2/wal/wals/8001930a232d4925bc5b11b4e3a390c3
I20260809 09:36:42.289361 16139 ts_tablet_manager.cc:1939] T 8001930a232d4925bc5b11b4e3a390c3 P fd74d537218d4dfa83b73820ef992eb0: Deleting consensus metadata
I20260809 09:36:42.290551 15333 catalog_manager.cc:4841] TS fd74d537218d4dfa83b73820ef992eb0 (127.20.125.131:40895): tablet 8001930a232d4925bc5b11b4e3a390c3 (table test-table [id=51eed9cc72f846a8b8e58c96d067870b]) successfully deleted
W20260809 09:36:42.294689 15328 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.20.125.129:45767 (error 108)
I20260809 09:36:42.294836 20982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu with pid 15656
I20260809 09:36:42.299409 16100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:36:42.297102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.129:45767 (local address 127.0.0.1:56380)
0809 09:36:42.299240 (+ 2138us) negotiation.cc:107] Waiting for socket to connect
0809 09:36:42.299317 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.129:45767: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2039}
W20260809 09:36:42.300485 15328 catalog_manager.cc:4584] TS 3f409f7879d442e292cf8a17d88ba28e (127.20.125.129:45767): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 8001930a232d4925bc5b11b4e3a390c3: Network error: Client connection negotiation failed: client connection to 127.20.125.129:45767: connect: Connection refused (error 111)
I20260809 09:36:42.307073 20982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu with pid 15863
I20260809 09:36:42.321343 20982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu with pid 15298
2026-08-09T09:36:42Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2578 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-09T09:36:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T09:36:42Z Disabled control of system clock
I20260809 09:36:42.359642 20982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
/tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786267941944493-20982-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786267941944493-20982-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786267941944493-20982-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786267941944493-20982-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.125.190:36719
--webserver_interface=127.20.125.190
--webserver_port=0
--builtin_ntp_servers=127.20.125.148:33971
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.125.190:36719
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 09:36:42.472301 16153 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 09:36:42.472579 16153 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 09:36:42.472615 16153 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 09:36:42.476595 16153 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 09:36:42.476653 16153 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 09:36:42.476670 16153 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 09:36:42.476686 16153 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 09:36:42.476707 16153 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 09:36:42.481319 16153 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.125.148:33971
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786267941944493-20982-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786267941944493-20982-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.125.190:36719
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.125.190:36719
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786267941944493-20982-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.125.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16153
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786267941944493-20982-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 08:38:55 UTC on ccycloud.quasar-njjleu.root.comops.site
build id 4575935
I20260809 09:36:42.482333 16153 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 09:36:42.483698 16153 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 09:36:42.493010 16166 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:36:42.493112 16163 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:36:42.493094 16153 server_base.cc:1034] running on GCE node
W20260809 09:36:42.493027 16164 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:36:42.493920 16153 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 09:36:42.495082 16153 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 09:36:42.496269 16153 hybrid_clock.cc:648] HybridClock initialized: now 1786268202496240 us; error 42 us; skew 500 ppm
I20260809 09:36:42.498183 16153 webserver.cc:460] Webserver started at http://127.20.125.190:40233/ using document root <none> and password file <none>
I20260809 09:36:42.498854 16153 fs_manager.cc:359] Metadata directory not provided
I20260809 09:36:42.498922 16153 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 09:36:42.499158 16153 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 09:36:42.501277 16153 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786267941944493-20982-0/minicluster-data/master-0/data/instance:
uuid: "929bf85b5e594d22bbcc292dd6267d50"
format_stamp: "Formatted at 2026-08-09 09:36:42 on dist-test-slave-7kzw"
server_key: "538f2dc01afc77c5633d9d5fc440cc75"
server_key_iv: "5561e578d7511577ba23cc16c717372e"
server_key_version: "encryptionkey@0"
I20260809 09:36:42.501856 16153 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786267941944493-20982-0/minicluster-data/master-0/wal/instance:
uuid: "929bf85b5e594d22bbcc292dd6267d50"
format_stamp: "Formatted at 2026-08-09 09:36:42 on dist-test-slave-7kzw"
server_key: "538f2dc01afc77c5633d9d5fc440cc75"
server_key_iv: "5561e578d7511577ba23cc16c717372e"
server_key_version: "encryptionkey@0"
I20260809 09:36:42.505587 16153 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.004s
I20260809 09:36:42.508070 16175 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 09:36:42.509361 16153 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 09:36:42.509505 16153 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786267941944493-20982-0/minicluster-data/master-0/data,/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786267941944493-20982-0/minicluster-data/master-0/wal
uuid: "929bf85b5e594d22bbcc292dd6267d50"
format_stamp: "Formatted at 2026-08-09 09:36:42 on dist-test-slave-7kzw"
server_key: "538f2dc01afc77c5633d9d5fc440cc75"
server_key_iv: "5561e578d7511577ba23cc16c717372e"
server_key_version: "encryptionkey@0"
I20260809 09:36:42.509617 16153 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786267941944493-20982-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786267941944493-20982-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786267941944493-20982-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 09:36:42.535082 16153 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 09:36:42.535804 16153 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 09:36:42.536010 16153 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 09:36:42.543437 16153 rpc_server.cc:307] RPC server started. Bound to: 127.20.125.190:36719
I20260809 09:36:42.543459 16254 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.125.190:36719 every 8 connection(s)
I20260809 09:36:42.544381 16153 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786267941944493-20982-0/minicluster-data/master-0/data/info.pb
I20260809 09:36:42.545876 20982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu as pid 16153
I20260809 09:36:42.546013 20982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786267941944493-20982-0/minicluster-data/master-0/wal/instance
I20260809 09:36:42.546372 20982 external_mini_cluster.cc:1428] Setting key 79a507ea30d65def4917b775ee6ae65f
I20260809 09:36:42.548228 16255 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 09:36:42.553768 16255 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 929bf85b5e594d22bbcc292dd6267d50: Bootstrap starting.
I20260809 09:36:42.556092 16258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:36:42.547845 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40362 (local address 127.20.125.190:36719)
0809 09:36:42.548555 (+ 710us) server_negotiation.cc:206] Beginning negotiation
0809 09:36:42.548571 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 09:36:42.548607 (+ 36us) server_negotiation.cc:407] Connection header received
0809 09:36:42.549251 (+ 644us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:36:42.549267 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:36:42.549710 (+ 443us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:36:42.550069 (+ 359us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:36:42.550539 (+ 470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:36:42.551547 (+ 1008us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:36:42.552490 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:36:42.552962 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:36:42.553144 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:36:42.554080 (+ 936us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:36:42.554086 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:36:42.554102 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 09:36:42.554601 (+ 499us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:36:42.554673 (+ 72us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 09:36:42.554700 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:36:42.554809 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0809 09:36:42.554914 (+ 105us) server_negotiation.cc:299] Negotiation successful
0809 09:36:42.555207 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":325,"thread_start_us":167,"threads_started":1}
I20260809 09:36:42.556452 16255 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 929bf85b5e594d22bbcc292dd6267d50: Neither blocks nor log segments found. Creating new log.
I20260809 09:36:42.557319 16255 log.cc:824] T 00000000000000000000000000000000 P 929bf85b5e594d22bbcc292dd6267d50: Log is configured to *not* fsync() on all Append() calls
I20260809 09:36:42.559367 16255 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 929bf85b5e594d22bbcc292dd6267d50: No bootstrap required, opened a new log
I20260809 09:36:42.562876 16255 raft_consensus.cc:348] T 00000000000000000000000000000000 P 929bf85b5e594d22bbcc292dd6267d50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "929bf85b5e594d22bbcc292dd6267d50" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 36719 } }
I20260809 09:36:42.563081 16255 raft_consensus.cc:374] T 00000000000000000000000000000000 P 929bf85b5e594d22bbcc292dd6267d50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 09:36:42.563158 16255 raft_consensus.cc:729] T 00000000000000000000000000000000 P 929bf85b5e594d22bbcc292dd6267d50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 929bf85b5e594d22bbcc292dd6267d50, State: Initialized, Role: FOLLOWER
I20260809 09:36:42.563542 16255 consensus_queue.cc:260] T 00000000000000000000000000000000 P 929bf85b5e594d22bbcc292dd6267d50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "929bf85b5e594d22bbcc292dd6267d50" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 36719 } }
I20260809 09:36:42.563668 16255 raft_consensus.cc:388] T 00000000000000000000000000000000 P 929bf85b5e594d22bbcc292dd6267d50 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 09:36:42.563750 16255 raft_consensus.cc:482] T 00000000000000000000000000000000 P 929bf85b5e594d22bbcc292dd6267d50 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 09:36:42.563854 16255 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 929bf85b5e594d22bbcc292dd6267d50 [term 0 FOLLOWER]: Advancing to term 1
I20260809 09:36:42.564718 16255 raft_consensus.cc:504] T 00000000000000000000000000000000 P 929bf85b5e594d22bbcc292dd6267d50 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "929bf85b5e594d22bbcc292dd6267d50" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 36719 } }
I20260809 09:36:42.565091 16255 leader_election.cc:302] T 00000000000000000000000000000000 P 929bf85b5e594d22bbcc292dd6267d50 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 929bf85b5e594d22bbcc292dd6267d50; no voters:
I20260809 09:36:42.565476 16255 leader_election.cc:288] T 00000000000000000000000000000000 P 929bf85b5e594d22bbcc292dd6267d50 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 09:36:42.565548 16261 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 929bf85b5e594d22bbcc292dd6267d50 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 09:36:42.565765 16261 raft_consensus.cc:686] T 00000000000000000000000000000000 P 929bf85b5e594d22bbcc292dd6267d50 [term 1 LEADER]: Becoming Leader. State: Replica: 929bf85b5e594d22bbcc292dd6267d50, State: Running, Role: LEADER
I20260809 09:36:42.566102 16261 consensus_queue.cc:237] T 00000000000000000000000000000000 P 929bf85b5e594d22bbcc292dd6267d50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "929bf85b5e594d22bbcc292dd6267d50" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 36719 } }
I20260809 09:36:42.566792 16255 sys_catalog.cc:564] T 00000000000000000000000000000000 P 929bf85b5e594d22bbcc292dd6267d50 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 09:36:42.567869 16263 sys_catalog.cc:455] T 00000000000000000000000000000000 P 929bf85b5e594d22bbcc292dd6267d50 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 929bf85b5e594d22bbcc292dd6267d50. Latest consensus state: current_term: 1 leader_uuid: "929bf85b5e594d22bbcc292dd6267d50" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "929bf85b5e594d22bbcc292dd6267d50" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 36719 } } }
I20260809 09:36:42.567907 16264 sys_catalog.cc:455] T 00000000000000000000000000000000 P 929bf85b5e594d22bbcc292dd6267d50 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "929bf85b5e594d22bbcc292dd6267d50" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "929bf85b5e594d22bbcc292dd6267d50" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 36719 } } }
I20260809 09:36:42.567997 16263 sys_catalog.cc:458] T 00000000000000000000000000000000 P 929bf85b5e594d22bbcc292dd6267d50 [sys.catalog]: This master's current role is: LEADER
I20260809 09:36:42.568011 16264 sys_catalog.cc:458] T 00000000000000000000000000000000 P 929bf85b5e594d22bbcc292dd6267d50 [sys.catalog]: This master's current role is: LEADER
I20260809 09:36:42.568395 16275 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 09:36:42.571146 16275 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 09:36:42.576200 16275 catalog_manager.cc:1347] Generated new cluster ID: 2da774ecdad040bead21b76da3614bc0
I20260809 09:36:42.576267 16275 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 09:36:42.590961 16275 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 09:36:42.592135 16275 catalog_manager.cc:1504] Loading token signing keys...
I20260809 09:36:42.597868 16275 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 929bf85b5e594d22bbcc292dd6267d50: Generated new TSK 0
I20260809 09:36:42.598634 16275 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 09:36:42.610206 20982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
/tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786267941944493-20982-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786267941944493-20982-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786267941944493-20982-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786267941944493-20982-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.125.129:0
--local_ip_for_outbound_sockets=127.20.125.129
--webserver_interface=127.20.125.129
--webserver_port=0
--tserver_master_addrs=127.20.125.190:36719
--builtin_ntp_servers=127.20.125.148:33971
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 09:36:42.724412 16293 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 09:36:42.724684 16293 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 09:36:42.724722 16293 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 09:36:42.728858 16293 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 09:36:42.728921 16293 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 09:36:42.728976 16293 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.125.129
I20260809 09:36:42.733811 16293 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.125.148:33971
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786267941944493-20982-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786267941944493-20982-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.125.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786267941944493-20982-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.125.129
--webserver_port=0
--tserver_master_addrs=127.20.125.190:36719
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16293
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.125.129
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786267941944493-20982-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 08:38:55 UTC on ccycloud.quasar-njjleu.root.comops.site
build id 4575935
I20260809 09:36:42.735014 16293 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 09:36:42.736438 16293 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 09:36:42.746749 16305 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:36:42.746868 16293 server_base.cc:1034] running on GCE node
W20260809 09:36:42.746767 16304 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:36:42.746760 16307 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:36:42.747589 16293 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 09:36:42.748273 16293 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 09:36:42.749476 16293 hybrid_clock.cc:648] HybridClock initialized: now 1786268202749448 us; error 39 us; skew 500 ppm
I20260809 09:36:42.751554 16293 webserver.cc:460] Webserver started at http://127.20.125.129:40189/ using document root <none> and password file <none>
I20260809 09:36:42.752170 16293 fs_manager.cc:359] Metadata directory not provided
I20260809 09:36:42.752269 16293 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 09:36:42.752516 16293 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 09:36:42.754683 16293 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786267941944493-20982-0/minicluster-data/ts-0/data/instance:
uuid: "6afb3dca1aed41e0b3a4392f7fa091f9"
format_stamp: "Formatted at 2026-08-09 09:36:42 on dist-test-slave-7kzw"
server_key: "b860a3354b7ac9fac3fef895b4bdd001"
server_key_iv: "1f2d11f1fe99128661bbe43d51c24111"
server_key_version: "encryptionkey@0"
I20260809 09:36:42.755255 16293 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786267941944493-20982-0/minicluster-data/ts-0/wal/instance:
uuid: "6afb3dca1aed41e0b3a4392f7fa091f9"
format_stamp: "Formatted at 2026-08-09 09:36:42 on dist-test-slave-7kzw"
server_key: "b860a3354b7ac9fac3fef895b4bdd001"
server_key_iv: "1f2d11f1fe99128661bbe43d51c24111"
server_key_version: "encryptionkey@0"
I20260809 09:36:42.759093 16293 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260809 09:36:42.761518 16315 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 09:36:42.762825 16293 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 09:36:42.762960 16293 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786267941944493-20982-0/minicluster-data/ts-0/data,/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786267941944493-20982-0/minicluster-data/ts-0/wal
uuid: "6afb3dca1aed41e0b3a4392f7fa091f9"
format_stamp: "Formatted at 2026-08-09 09:36:42 on dist-test-slave-7kzw"
server_key: "b860a3354b7ac9fac3fef895b4bdd001"
server_key_iv: "1f2d11f1fe99128661bbe43d51c24111"
server_key_version: "encryptionkey@0"
I20260809 09:36:42.763073 16293 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786267941944493-20982-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786267941944493-20982-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786267941944493-20982-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 09:36:42.780290 16293 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 09:36:42.781052 16293 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 09:36:42.781280 16293 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 09:36:42.781901 16293 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 09:36:42.782997 16293 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 09:36:42.783077 16293 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 09:36:42.783151 16293 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 09:36:42.783200 16293 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 09:36:42.793476 16293 rpc_server.cc:307] RPC server started. Bound to: 127.20.125.129:34497
I20260809 09:36:42.793516 16490 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.125.129:34497 every 8 connection(s)
I20260809 09:36:42.794836 16293 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786267941944493-20982-0/minicluster-data/ts-0/data/info.pb
I20260809 09:36:42.797941 20982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu as pid 16293
I20260809 09:36:42.798112 20982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786267941944493-20982-0/minicluster-data/ts-0/wal/instance
I20260809 09:36:42.798494 20982 external_mini_cluster.cc:1428] Setting key 924a891f6150e3d0e9d4d2bf9e97fa2b
I20260809 09:36:42.800894 20982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
/tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786267941944493-20982-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786267941944493-20982-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786267941944493-20982-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786267941944493-20982-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.125.130:0
--local_ip_for_outbound_sockets=127.20.125.130
--webserver_interface=127.20.125.130
--webserver_port=0
--tserver_master_addrs=127.20.125.190:36719
--builtin_ntp_servers=127.20.125.148:33971
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260809 09:36:42.806408 16258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:36:42.797040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.129:47429 (local address 127.20.125.190:36719)
0809 09:36:42.797207 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0809 09:36:42.797211 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 09:36:42.798505 (+ 1294us) server_negotiation.cc:407] Connection header received
0809 09:36:42.800056 (+ 1551us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:36:42.800061 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:36:42.800123 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:36:42.800234 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:36:42.801553 (+ 1319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:36:42.802138 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:36:42.803754 (+ 1616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:36:42.804197 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:36:42.804356 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:36:42.805655 (+ 1299us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:36:42.805670 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:36:42.805677 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 09:36:42.805699 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:36:42.805730 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 09:36:42.805744 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:36:42.805838 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0809 09:36:42.806048 (+ 210us) server_negotiation.cc:299] Negotiation successful
0809 09:36:42.806157 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260809 09:36:42.807258 16493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:36:42.797462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:36719 (local address 127.20.125.129:47429)
0809 09:36:42.798296 (+ 834us) negotiation.cc:107] Waiting for socket to connect
0809 09:36:42.798355 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0809 09:36:42.799841 (+ 1486us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:36:42.800457 (+ 616us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:36:42.800466 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:36:42.800580 (+ 114us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:36:42.801381 (+ 801us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:36:42.801406 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:36:42.802303 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:36:42.802311 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:36:42.803593 (+ 1282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:36:42.803604 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:36:42.804393 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:36:42.804397 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:36:42.804554 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:36:42.805348 (+ 794us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 09:36:42.805350 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:36:42.805395 (+ 45us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 09:36:42.805470 (+ 75us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 09:36:42.805865 (+ 395us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 09:36:42.805874 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 09:36:42.805880 (+ 6us) client_negotiation.cc:769] Sending connection context
0809 09:36:42.806004 (+ 124us) client_negotiation.cc:240] Negotiation successful
0809 09:36:42.806256 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":324,"thread_start_us":161,"threads_started":1}
I20260809 09:36:42.808846 16491 heartbeater.cc:347] Connected to a master server at 127.20.125.190:36719
I20260809 09:36:42.809118 16491 heartbeater.cc:464] Registering TS with master...
I20260809 09:36:42.809693 16491 heartbeater.cc:511] Master 127.20.125.190:36719 requested a full tablet report, sending...
I20260809 09:36:42.811352 16203 ts_manager.cc:194] Registered new tserver with Master: 6afb3dca1aed41e0b3a4392f7fa091f9 (127.20.125.129:34497)
I20260809 09:36:42.812548 16203 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.20.125.129:47429
W20260809 09:36:42.916716 16494 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 09:36:42.916968 16494 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 09:36:42.917006 16494 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 09:36:42.921144 16494 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 09:36:42.921214 16494 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 09:36:42.921298 16494 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.125.130
I20260809 09:36:42.926133 16494 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.125.148:33971
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786267941944493-20982-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786267941944493-20982-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.125.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786267941944493-20982-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.125.130
--webserver_port=0
--tserver_master_addrs=127.20.125.190:36719
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16494
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.125.130
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786267941944493-20982-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 08:38:55 UTC on ccycloud.quasar-njjleu.root.comops.site
build id 4575935
I20260809 09:36:42.927237 16494 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 09:36:42.928642 16494 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 09:36:42.938520 16502 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:36:42.938499 16504 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:36:42.938494 16499 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:36:42.938930 16494 server_base.cc:1034] running on GCE node
I20260809 09:36:42.939339 16494 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 09:36:42.939993 16494 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 09:36:42.941186 16494 hybrid_clock.cc:648] HybridClock initialized: now 1786268202941172 us; error 34 us; skew 500 ppm
I20260809 09:36:42.943202 16494 webserver.cc:460] Webserver started at http://127.20.125.130:37017/ using document root <none> and password file <none>
I20260809 09:36:42.943790 16494 fs_manager.cc:359] Metadata directory not provided
I20260809 09:36:42.943887 16494 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 09:36:42.944132 16494 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 09:36:42.946194 16494 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786267941944493-20982-0/minicluster-data/ts-1/data/instance:
uuid: "200658f84e7d45b699327eb83c2159cb"
format_stamp: "Formatted at 2026-08-09 09:36:42 on dist-test-slave-7kzw"
server_key: "d39ea47b8ce9c0e38ef823677de6da31"
server_key_iv: "14151c24955e3b709ca05ecef3d28956"
server_key_version: "encryptionkey@0"
I20260809 09:36:42.946799 16494 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786267941944493-20982-0/minicluster-data/ts-1/wal/instance:
uuid: "200658f84e7d45b699327eb83c2159cb"
format_stamp: "Formatted at 2026-08-09 09:36:42 on dist-test-slave-7kzw"
server_key: "d39ea47b8ce9c0e38ef823677de6da31"
server_key_iv: "14151c24955e3b709ca05ecef3d28956"
server_key_version: "encryptionkey@0"
I20260809 09:36:42.950656 16494 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260809 09:36:42.953188 16511 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 09:36:42.954393 16494 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 09:36:42.954550 16494 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786267941944493-20982-0/minicluster-data/ts-1/data,/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786267941944493-20982-0/minicluster-data/ts-1/wal
uuid: "200658f84e7d45b699327eb83c2159cb"
format_stamp: "Formatted at 2026-08-09 09:36:42 on dist-test-slave-7kzw"
server_key: "d39ea47b8ce9c0e38ef823677de6da31"
server_key_iv: "14151c24955e3b709ca05ecef3d28956"
server_key_version: "encryptionkey@0"
I20260809 09:36:42.954665 16494 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786267941944493-20982-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786267941944493-20982-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786267941944493-20982-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 09:36:42.976985 16494 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 09:36:42.977725 16494 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 09:36:42.977936 16494 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 09:36:42.978598 16494 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 09:36:42.979581 16494 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 09:36:42.979657 16494 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 09:36:42.979743 16494 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 09:36:42.979799 16494 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 09:36:42.989043 16494 rpc_server.cc:307] RPC server started. Bound to: 127.20.125.130:39855
I20260809 09:36:42.989064 16672 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.125.130:39855 every 8 connection(s)
I20260809 09:36:42.989959 16494 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786267941944493-20982-0/minicluster-data/ts-1/data/info.pb
I20260809 09:36:42.999221 16258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:36:42.991859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.130:54861 (local address 127.20.125.190:36719)
0809 09:36:42.992018 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0809 09:36:42.992023 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 09:36:42.993074 (+ 1051us) server_negotiation.cc:407] Connection header received
0809 09:36:42.994458 (+ 1384us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:36:42.994460 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:36:42.994529 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:36:42.994604 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:36:42.995461 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:36:42.996001 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:36:42.997292 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:36:42.997503 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:36:42.997586 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:36:42.998758 (+ 1172us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:36:42.998761 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:36:42.998763 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 09:36:42.998775 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:36:42.998793 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 09:36:42.998801 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:36:42.998861 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0809 09:36:42.999043 (+ 182us) server_negotiation.cc:299] Negotiation successful
0809 09:36:42.999103 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 09:36:42.999909 20982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu as pid 16494
I20260809 09:36:43.000047 20982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786267941944493-20982-0/minicluster-data/ts-1/wal/instance
I20260809 09:36:43.000240 16677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:36:42.992268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:36719 (local address 127.20.125.130:54861)
0809 09:36:42.992929 (+ 661us) negotiation.cc:107] Waiting for socket to connect
0809 09:36:42.992969 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 09:36:42.994261 (+ 1292us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:36:42.994773 (+ 512us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:36:42.994779 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:36:42.994858 (+ 79us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:36:42.995326 (+ 468us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:36:42.995338 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:36:42.996150 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:36:42.996153 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:36:42.997172 (+ 1019us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:36:42.997182 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:36:42.997700 (+ 518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:36:42.997703 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:36:42.997852 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:36:42.998457 (+ 605us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 09:36:42.998474 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:36:42.998514 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 09:36:42.998576 (+ 62us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 09:36:42.998878 (+ 302us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 09:36:42.998885 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 09:36:42.998889 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 09:36:42.998984 (+ 95us) client_negotiation.cc:240] Negotiation successful
0809 09:36:42.999198 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":268,"thread_start_us":113,"threads_started":1}
I20260809 09:36:43.000429 20982 external_mini_cluster.cc:1428] Setting key f9b48e51a6c3eac9a4d2094d57ccf01b
I20260809 09:36:43.001425 16673 heartbeater.cc:347] Connected to a master server at 127.20.125.190:36719
I20260809 09:36:43.001683 16673 heartbeater.cc:464] Registering TS with master...
I20260809 09:36:43.002259 16673 heartbeater.cc:511] Master 127.20.125.190:36719 requested a full tablet report, sending...
I20260809 09:36:43.002646 20982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
/tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786267941944493-20982-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786267941944493-20982-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786267941944493-20982-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786267941944493-20982-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.125.131:0
--local_ip_for_outbound_sockets=127.20.125.131
--webserver_interface=127.20.125.131
--webserver_port=0
--tserver_master_addrs=127.20.125.190:36719
--builtin_ntp_servers=127.20.125.148:33971
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260809 09:36:43.003412 16203 ts_manager.cc:194] Registered new tserver with Master: 200658f84e7d45b699327eb83c2159cb (127.20.125.130:39855)
I20260809 09:36:43.004076 16203 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.20.125.130:54861
W20260809 09:36:43.121018 16678 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 09:36:43.121337 16678 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 09:36:43.121410 16678 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 09:36:43.125430 16678 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 09:36:43.125522 16678 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 09:36:43.125617 16678 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.125.131
I20260809 09:36:43.131578 16678 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.125.148:33971
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786267941944493-20982-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786267941944493-20982-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.125.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786267941944493-20982-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.125.131
--webserver_port=0
--tserver_master_addrs=127.20.125.190:36719
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16678
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.125.131
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786267941944493-20982-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 08:38:55 UTC on ccycloud.quasar-njjleu.root.comops.site
build id 4575935
I20260809 09:36:43.132849 16678 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 09:36:43.134268 16678 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 09:36:43.144116 16678 server_base.cc:1034] running on GCE node
W20260809 09:36:43.144042 16686 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:36:43.144117 16690 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:36:43.144019 16684 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:36:43.144734 16678 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 09:36:43.145401 16678 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 09:36:43.146595 16678 hybrid_clock.cc:648] HybridClock initialized: now 1786268203146532 us; error 39 us; skew 500 ppm
I20260809 09:36:43.148464 16678 webserver.cc:460] Webserver started at http://127.20.125.131:38101/ using document root <none> and password file <none>
I20260809 09:36:43.149091 16678 fs_manager.cc:359] Metadata directory not provided
I20260809 09:36:43.149194 16678 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 09:36:43.149427 16678 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 09:36:43.151765 16678 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786267941944493-20982-0/minicluster-data/ts-2/data/instance:
uuid: "111d8fe1f13d42eaabc559d89832bc6d"
format_stamp: "Formatted at 2026-08-09 09:36:43 on dist-test-slave-7kzw"
server_key: "2c96c937bafe6eae0da940b60f1bcd78"
server_key_iv: "23e6d0e3b2bd1f5358a7701343e6a1db"
server_key_version: "encryptionkey@0"
I20260809 09:36:43.152379 16678 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786267941944493-20982-0/minicluster-data/ts-2/wal/instance:
uuid: "111d8fe1f13d42eaabc559d89832bc6d"
format_stamp: "Formatted at 2026-08-09 09:36:43 on dist-test-slave-7kzw"
server_key: "2c96c937bafe6eae0da940b60f1bcd78"
server_key_iv: "23e6d0e3b2bd1f5358a7701343e6a1db"
server_key_version: "encryptionkey@0"
I20260809 09:36:43.156333 16678 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 09:36:43.159235 16699 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 09:36:43.160712 16678 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 09:36:43.160820 16678 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786267941944493-20982-0/minicluster-data/ts-2/data,/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786267941944493-20982-0/minicluster-data/ts-2/wal
uuid: "111d8fe1f13d42eaabc559d89832bc6d"
format_stamp: "Formatted at 2026-08-09 09:36:43 on dist-test-slave-7kzw"
server_key: "2c96c937bafe6eae0da940b60f1bcd78"
server_key_iv: "23e6d0e3b2bd1f5358a7701343e6a1db"
server_key_version: "encryptionkey@0"
I20260809 09:36:43.160965 16678 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786267941944493-20982-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786267941944493-20982-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786267941944493-20982-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 09:36:43.179726 16678 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 09:36:43.180545 16678 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 09:36:43.180770 16678 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 09:36:43.181432 16678 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 09:36:43.182418 16678 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 09:36:43.182533 16678 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 09:36:43.182581 16678 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 09:36:43.182605 16678 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 09:36:43.191759 16678 rpc_server.cc:307] RPC server started. Bound to: 127.20.125.131:40839
I20260809 09:36:43.191839 16874 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.125.131:40839 every 8 connection(s)
I20260809 09:36:43.192730 16678 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786267941944493-20982-0/minicluster-data/ts-2/data/info.pb
I20260809 09:36:43.200680 20982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu as pid 16678
I20260809 09:36:43.200858 20982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786267941944493-20982-0/minicluster-data/ts-2/wal/instance
I20260809 09:36:43.201213 20982 external_mini_cluster.cc:1428] Setting key 06bce31d90d4448427836a9c2531e752
I20260809 09:36:43.203440 16258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:36:43.194880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.131:39795 (local address 127.20.125.190:36719)
0809 09:36:43.195075 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0809 09:36:43.195080 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 09:36:43.196203 (+ 1123us) server_negotiation.cc:407] Connection header received
0809 09:36:43.197757 (+ 1554us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:36:43.197761 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:36:43.197825 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:36:43.197944 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:36:43.198971 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:36:43.199560 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:36:43.200873 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:36:43.201180 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:36:43.201312 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:36:43.202793 (+ 1481us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:36:43.202798 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:36:43.202801 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 09:36:43.202815 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:36:43.202842 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 09:36:43.202854 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:36:43.202956 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0809 09:36:43.203221 (+ 265us) server_negotiation.cc:299] Negotiation successful
0809 09:36:43.203287 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260809 09:36:43.204463 16878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:36:43.195275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:36719 (local address 127.20.125.131:39795)
0809 09:36:43.196011 (+ 736us) negotiation.cc:107] Waiting for socket to connect
0809 09:36:43.196075 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0809 09:36:43.197522 (+ 1447us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:36:43.198134 (+ 612us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:36:43.198142 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:36:43.198228 (+ 86us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:36:43.198815 (+ 587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:36:43.198829 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:36:43.199701 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:36:43.199712 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:36:43.200682 (+ 970us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:36:43.200692 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:36:43.201356 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:36:43.201361 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:36:43.201570 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:36:43.202401 (+ 831us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 09:36:43.202403 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:36:43.202466 (+ 63us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 09:36:43.202611 (+ 145us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 09:36:43.203019 (+ 408us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 09:36:43.203029 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 09:36:43.203036 (+ 7us) client_negotiation.cc:769] Sending connection context
0809 09:36:43.203192 (+ 156us) client_negotiation.cc:240] Negotiation successful
0809 09:36:43.203481 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":315,"thread_start_us":118,"threads_started":1}
I20260809 09:36:43.205650 16875 heartbeater.cc:347] Connected to a master server at 127.20.125.190:36719
I20260809 09:36:43.205917 16875 heartbeater.cc:464] Registering TS with master...
I20260809 09:36:43.206554 16875 heartbeater.cc:511] Master 127.20.125.190:36719 requested a full tablet report, sending...
I20260809 09:36:43.207719 16203 ts_manager.cc:194] Registered new tserver with Master: 111d8fe1f13d42eaabc559d89832bc6d (127.20.125.131:40839)
I20260809 09:36:43.208292 16203 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.20.125.131:39795
I20260809 09:36:43.216722 20982 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 09:36:43.223271 16258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:36:43.218969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40378 (local address 127.20.125.190:36719)
0809 09:36:43.219118 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0809 09:36:43.219123 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 09:36:43.219321 (+ 198us) server_negotiation.cc:407] Connection header received
0809 09:36:43.219492 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:36:43.219495 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:36:43.219548 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:36:43.219637 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:36:43.219983 (+ 346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:36:43.220515 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:36:43.221388 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:36:43.221614 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:36:43.221728 (+ 114us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:36:43.222733 (+ 1005us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:36:43.222742 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:36:43.222744 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 09:36:43.222755 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:36:43.222774 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 09:36:43.222783 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:36:43.222924 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0809 09:36:43.223076 (+ 152us) server_negotiation.cc:299] Negotiation successful
0809 09:36:43.223114 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260809 09:36:43.226361 16203 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:40378:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 09:36:43.229401 16203 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 09:36:43.244903 16892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:36:43.239268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52134 (local address 127.20.125.130:39855)
0809 09:36:43.239806 (+ 538us) server_negotiation.cc:206] Beginning negotiation
0809 09:36:43.239821 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 09:36:43.239846 (+ 25us) server_negotiation.cc:407] Connection header received
0809 09:36:43.239925 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:36:43.239930 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:36:43.240107 (+ 177us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:36:43.240262 (+ 155us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 09:36:43.240514 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:36:43.241620 (+ 1106us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:36:43.243530 (+ 1910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:36:43.244113 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:36:43.244274 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:36:43.244463 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0809 09:36:43.244608 (+ 145us) server_negotiation.cc:299] Negotiation successful
0809 09:36:43.244720 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":435,"thread_start_us":144,"threads_started":1}
I20260809 09:36:43.244899 16889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:36:43.238193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.130:39855 (local address 127.0.0.1:52134)
0809 09:36:43.239051 (+ 858us) negotiation.cc:107] Waiting for socket to connect
0809 09:36:43.239067 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 09:36:43.239165 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:36:43.240262 (+ 1097us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:36:43.240265 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:36:43.240289 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 09:36:43.240381 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:36:43.240387 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:36:43.241724 (+ 1337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:36:43.241727 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:36:43.243374 (+ 1647us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:36:43.243381 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:36:43.244262 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:36:43.244266 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:36:43.244382 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:36:43.244423 (+ 41us) client_negotiation.cc:769] Sending connection context
0809 09:36:43.244571 (+ 148us) client_negotiation.cc:240] Negotiation successful
0809 09:36:43.244721 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":782,"mutex_wait_us":306,"thread_start_us":113,"threads_started":1}
I20260809 09:36:43.245415 16891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:36:43.238940 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.131:40839 (local address 127.0.0.1:39900)
0809 09:36:43.239771 (+ 831us) negotiation.cc:107] Waiting for socket to connect
0809 09:36:43.239785 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 09:36:43.239877 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:36:43.240599 (+ 722us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:36:43.240602 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:36:43.240624 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 09:36:43.240821 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:36:43.240830 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:36:43.242222 (+ 1392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:36:43.242225 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:36:43.243208 (+ 983us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:36:43.243218 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:36:43.244533 (+ 1315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:36:43.244536 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:36:43.244772 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:36:43.244811 (+ 39us) client_negotiation.cc:769] Sending connection context
0809 09:36:43.244879 (+ 68us) client_negotiation.cc:240] Negotiation successful
0809 09:36:43.245277 (+ 398us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":770,"mutex_wait_us":412,"thread_start_us":94,"threads_started":1}
I20260809 09:36:43.245985 16890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:36:43.238322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45566 (local address 127.20.125.129:34497)
0809 09:36:43.239125 (+ 803us) server_negotiation.cc:206] Beginning negotiation
0809 09:36:43.239141 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 09:36:43.239170 (+ 29us) server_negotiation.cc:407] Connection header received
0809 09:36:43.239261 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:36:43.239267 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:36:43.239495 (+ 228us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:36:43.239684 (+ 189us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 09:36:43.240245 (+ 561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:36:43.241439 (+ 1194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:36:43.243503 (+ 2064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:36:43.244257 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:36:43.244363 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:36:43.244494 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0809 09:36:43.245702 (+ 1208us) server_negotiation.cc:299] Negotiation successful
0809 09:36:43.245810 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":686,"thread_start_us":185,"threads_started":1}
I20260809 09:36:43.246258 16888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:36:43.238179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.129:34497 (local address 127.0.0.1:45566)
0809 09:36:43.238536 (+ 357us) negotiation.cc:107] Waiting for socket to connect
0809 09:36:43.238587 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0809 09:36:43.238722 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:36:43.239892 (+ 1170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:36:43.239897 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:36:43.239920 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 09:36:43.240046 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:36:43.240052 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:36:43.241611 (+ 1559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:36:43.241613 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:36:43.243359 (+ 1746us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:36:43.243372 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:36:43.244544 (+ 1172us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:36:43.244547 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:36:43.244747 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:36:43.244806 (+ 59us) client_negotiation.cc:769] Sending connection context
0809 09:36:43.244879 (+ 73us) client_negotiation.cc:240] Negotiation successful
0809 09:36:43.244976 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"mutex_wait_us":135,"thread_start_us":73,"threads_started":1}
I20260809 09:36:43.246254 16894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:36:43.239386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39900 (local address 127.20.125.131:40839)
0809 09:36:43.240125 (+ 739us) server_negotiation.cc:206] Beginning negotiation
0809 09:36:43.240143 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0809 09:36:43.240169 (+ 26us) server_negotiation.cc:407] Connection header received
0809 09:36:43.240258 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:36:43.240263 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:36:43.240432 (+ 169us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:36:43.240608 (+ 176us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 09:36:43.240949 (+ 341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:36:43.242054 (+ 1105us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:36:43.243335 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:36:43.244150 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:36:43.244275 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:36:43.244462 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0809 09:36:43.245141 (+ 679us) server_negotiation.cc:299] Negotiation successful
0809 09:36:43.246084 (+ 943us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":632,"thread_start_us":137,"threads_started":1}
I20260809 09:36:43.248226 16578 tablet_service.cc:1467] Processing CreateTablet for tablet fca6c797a6e44a2db7b4003db8f27512 (DEFAULT_TABLE table=test-table [id=21dd47c22f2d4e2087fb280caa4c1bd8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 09:36:43.249388 16578 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fca6c797a6e44a2db7b4003db8f27512. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 09:36:43.249037 16777 tablet_service.cc:1467] Processing CreateTablet for tablet fca6c797a6e44a2db7b4003db8f27512 (DEFAULT_TABLE table=test-table [id=21dd47c22f2d4e2087fb280caa4c1bd8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 09:36:43.249037 16394 tablet_service.cc:1467] Processing CreateTablet for tablet fca6c797a6e44a2db7b4003db8f27512 (DEFAULT_TABLE table=test-table [id=21dd47c22f2d4e2087fb280caa4c1bd8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 09:36:43.250049 16394 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fca6c797a6e44a2db7b4003db8f27512. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 09:36:43.250049 16777 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fca6c797a6e44a2db7b4003db8f27512. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 09:36:43.256328 16895 tablet_bootstrap.cc:492] T fca6c797a6e44a2db7b4003db8f27512 P 200658f84e7d45b699327eb83c2159cb: Bootstrap starting.
I20260809 09:36:43.257695 16896 tablet_bootstrap.cc:492] T fca6c797a6e44a2db7b4003db8f27512 P 6afb3dca1aed41e0b3a4392f7fa091f9: Bootstrap starting.
I20260809 09:36:43.258810 16897 tablet_bootstrap.cc:492] T fca6c797a6e44a2db7b4003db8f27512 P 111d8fe1f13d42eaabc559d89832bc6d: Bootstrap starting.
I20260809 09:36:43.258972 16895 tablet_bootstrap.cc:654] T fca6c797a6e44a2db7b4003db8f27512 P 200658f84e7d45b699327eb83c2159cb: Neither blocks nor log segments found. Creating new log.
I20260809 09:36:43.259768 16895 log.cc:824] T fca6c797a6e44a2db7b4003db8f27512 P 200658f84e7d45b699327eb83c2159cb: Log is configured to *not* fsync() on all Append() calls
I20260809 09:36:43.259797 16896 tablet_bootstrap.cc:654] T fca6c797a6e44a2db7b4003db8f27512 P 6afb3dca1aed41e0b3a4392f7fa091f9: Neither blocks nor log segments found. Creating new log.
I20260809 09:36:43.260614 16896 log.cc:824] T fca6c797a6e44a2db7b4003db8f27512 P 6afb3dca1aed41e0b3a4392f7fa091f9: Log is configured to *not* fsync() on all Append() calls
I20260809 09:36:43.260903 16897 tablet_bootstrap.cc:654] T fca6c797a6e44a2db7b4003db8f27512 P 111d8fe1f13d42eaabc559d89832bc6d: Neither blocks nor log segments found. Creating new log.
I20260809 09:36:43.261669 16897 log.cc:824] T fca6c797a6e44a2db7b4003db8f27512 P 111d8fe1f13d42eaabc559d89832bc6d: Log is configured to *not* fsync() on all Append() calls
I20260809 09:36:43.262329 16895 tablet_bootstrap.cc:492] T fca6c797a6e44a2db7b4003db8f27512 P 200658f84e7d45b699327eb83c2159cb: No bootstrap required, opened a new log
I20260809 09:36:43.262568 16895 ts_tablet_manager.cc:1397] T fca6c797a6e44a2db7b4003db8f27512 P 200658f84e7d45b699327eb83c2159cb: Time spent bootstrapping tablet: real 0.006s user 0.006s sys 0.000s
I20260809 09:36:43.262923 16896 tablet_bootstrap.cc:492] T fca6c797a6e44a2db7b4003db8f27512 P 6afb3dca1aed41e0b3a4392f7fa091f9: No bootstrap required, opened a new log
I20260809 09:36:43.263165 16896 ts_tablet_manager.cc:1397] T fca6c797a6e44a2db7b4003db8f27512 P 6afb3dca1aed41e0b3a4392f7fa091f9: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.003s
I20260809 09:36:43.263466 16897 tablet_bootstrap.cc:492] T fca6c797a6e44a2db7b4003db8f27512 P 111d8fe1f13d42eaabc559d89832bc6d: No bootstrap required, opened a new log
I20260809 09:36:43.263656 16897 ts_tablet_manager.cc:1397] T fca6c797a6e44a2db7b4003db8f27512 P 111d8fe1f13d42eaabc559d89832bc6d: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260809 09:36:43.267684 16897 raft_consensus.cc:348] T fca6c797a6e44a2db7b4003db8f27512 P 111d8fe1f13d42eaabc559d89832bc6d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6afb3dca1aed41e0b3a4392f7fa091f9" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 34497 } } peers { permanent_uuid: "200658f84e7d45b699327eb83c2159cb" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 39855 } } peers { permanent_uuid: "111d8fe1f13d42eaabc559d89832bc6d" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 40839 } }
I20260809 09:36:43.268023 16897 raft_consensus.cc:374] T fca6c797a6e44a2db7b4003db8f27512 P 111d8fe1f13d42eaabc559d89832bc6d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 09:36:43.267877 16895 raft_consensus.cc:348] T fca6c797a6e44a2db7b4003db8f27512 P 200658f84e7d45b699327eb83c2159cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6afb3dca1aed41e0b3a4392f7fa091f9" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 34497 } } peers { permanent_uuid: "200658f84e7d45b699327eb83c2159cb" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 39855 } } peers { permanent_uuid: "111d8fe1f13d42eaabc559d89832bc6d" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 40839 } }
I20260809 09:36:43.268085 16897 raft_consensus.cc:729] T fca6c797a6e44a2db7b4003db8f27512 P 111d8fe1f13d42eaabc559d89832bc6d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 111d8fe1f13d42eaabc559d89832bc6d, State: Initialized, Role: FOLLOWER
I20260809 09:36:43.268143 16895 raft_consensus.cc:374] T fca6c797a6e44a2db7b4003db8f27512 P 200658f84e7d45b699327eb83c2159cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 09:36:43.268229 16895 raft_consensus.cc:729] T fca6c797a6e44a2db7b4003db8f27512 P 200658f84e7d45b699327eb83c2159cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 200658f84e7d45b699327eb83c2159cb, State: Initialized, Role: FOLLOWER
I20260809 09:36:43.268278 16896 raft_consensus.cc:348] T fca6c797a6e44a2db7b4003db8f27512 P 6afb3dca1aed41e0b3a4392f7fa091f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6afb3dca1aed41e0b3a4392f7fa091f9" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 34497 } } peers { permanent_uuid: "200658f84e7d45b699327eb83c2159cb" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 39855 } } peers { permanent_uuid: "111d8fe1f13d42eaabc559d89832bc6d" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 40839 } }
I20260809 09:36:43.268532 16896 raft_consensus.cc:374] T fca6c797a6e44a2db7b4003db8f27512 P 6afb3dca1aed41e0b3a4392f7fa091f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 09:36:43.268606 16896 raft_consensus.cc:729] T fca6c797a6e44a2db7b4003db8f27512 P 6afb3dca1aed41e0b3a4392f7fa091f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6afb3dca1aed41e0b3a4392f7fa091f9, State: Initialized, Role: FOLLOWER
I20260809 09:36:43.268576 16897 consensus_queue.cc:260] T fca6c797a6e44a2db7b4003db8f27512 P 111d8fe1f13d42eaabc559d89832bc6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6afb3dca1aed41e0b3a4392f7fa091f9" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 34497 } } peers { permanent_uuid: "200658f84e7d45b699327eb83c2159cb" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 39855 } } peers { permanent_uuid: "111d8fe1f13d42eaabc559d89832bc6d" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 40839 } }
I20260809 09:36:43.268725 16895 consensus_queue.cc:260] T fca6c797a6e44a2db7b4003db8f27512 P 200658f84e7d45b699327eb83c2159cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6afb3dca1aed41e0b3a4392f7fa091f9" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 34497 } } peers { permanent_uuid: "200658f84e7d45b699327eb83c2159cb" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 39855 } } peers { permanent_uuid: "111d8fe1f13d42eaabc559d89832bc6d" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 40839 } }
I20260809 09:36:43.269066 16896 consensus_queue.cc:260] T fca6c797a6e44a2db7b4003db8f27512 P 6afb3dca1aed41e0b3a4392f7fa091f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6afb3dca1aed41e0b3a4392f7fa091f9" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 34497 } } peers { permanent_uuid: "200658f84e7d45b699327eb83c2159cb" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 39855 } } peers { permanent_uuid: "111d8fe1f13d42eaabc559d89832bc6d" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 40839 } }
I20260809 09:36:43.269507 16673 heartbeater.cc:503] Master 127.20.125.190:36719 was elected leader, sending a full tablet report...
I20260809 09:36:43.269799 16895 ts_tablet_manager.cc:1428] T fca6c797a6e44a2db7b4003db8f27512 P 200658f84e7d45b699327eb83c2159cb: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260809 09:36:43.269965 16896 ts_tablet_manager.cc:1428] T fca6c797a6e44a2db7b4003db8f27512 P 6afb3dca1aed41e0b3a4392f7fa091f9: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260809 09:36:43.270238 16897 ts_tablet_manager.cc:1428] T fca6c797a6e44a2db7b4003db8f27512 P 111d8fe1f13d42eaabc559d89832bc6d: Time spent starting tablet: real 0.007s user 0.001s sys 0.005s
I20260809 09:36:43.270323 16491 heartbeater.cc:503] Master 127.20.125.190:36719 was elected leader, sending a full tablet report...
I20260809 09:36:43.270601 16875 heartbeater.cc:503] Master 127.20.125.190:36719 was elected leader, sending a full tablet report...
W20260809 09:36:43.296141 16492 tablet.cc:2369] T fca6c797a6e44a2db7b4003db8f27512 P 6afb3dca1aed41e0b3a4392f7fa091f9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 09:36:43.347144 16903 raft_consensus.cc:482] T fca6c797a6e44a2db7b4003db8f27512 P 200658f84e7d45b699327eb83c2159cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 09:36:43.347361 16903 raft_consensus.cc:504] T fca6c797a6e44a2db7b4003db8f27512 P 200658f84e7d45b699327eb83c2159cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6afb3dca1aed41e0b3a4392f7fa091f9" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 34497 } } peers { permanent_uuid: "200658f84e7d45b699327eb83c2159cb" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 39855 } } peers { permanent_uuid: "111d8fe1f13d42eaabc559d89832bc6d" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 40839 } }
I20260809 09:36:43.348524 16903 leader_election.cc:288] T fca6c797a6e44a2db7b4003db8f27512 P 200658f84e7d45b699327eb83c2159cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6afb3dca1aed41e0b3a4392f7fa091f9 (127.20.125.129:34497), 111d8fe1f13d42eaabc559d89832bc6d (127.20.125.131:40839)
I20260809 09:36:43.352560 16890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:36:43.349012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.130:34379 (local address 127.20.125.129:34497)
0809 09:36:43.349147 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0809 09:36:43.349152 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 09:36:43.349168 (+ 16us) server_negotiation.cc:407] Connection header received
0809 09:36:43.349242 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:36:43.349245 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:36:43.349309 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:36:43.349391 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 09:36:43.349753 (+ 362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:36:43.350391 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:36:43.351572 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:36:43.351974 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:36:43.352067 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:36:43.352146 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0809 09:36:43.352364 (+ 218us) server_negotiation.cc:299] Negotiation successful
0809 09:36:43.352435 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260809 09:36:43.352562 16894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:36:43.348936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.130:47751 (local address 127.20.125.131:40839)
0809 09:36:43.349082 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0809 09:36:43.349088 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 09:36:43.349430 (+ 342us) server_negotiation.cc:407] Connection header received
0809 09:36:43.349503 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:36:43.349506 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:36:43.349609 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:36:43.349715 (+ 106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 09:36:43.350020 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:36:43.350574 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:36:43.351609 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:36:43.352006 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:36:43.352143 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:36:43.352194 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0809 09:36:43.352375 (+ 181us) server_negotiation.cc:299] Negotiation successful
0809 09:36:43.352446 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260809 09:36:43.352561 16677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:36:43.348764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.129:34497 (local address 127.20.125.130:34379)
0809 09:36:43.348890 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0809 09:36:43.348907 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 09:36:43.349009 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:36:43.349456 (+ 447us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:36:43.349459 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:36:43.349484 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 09:36:43.349621 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:36:43.349632 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:36:43.350529 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:36:43.350532 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:36:43.351442 (+ 910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:36:43.351452 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:36:43.352101 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:36:43.352107 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:36:43.352220 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:36:43.352247 (+ 27us) client_negotiation.cc:769] Sending connection context
0809 09:36:43.352346 (+ 99us) client_negotiation.cc:240] Negotiation successful
0809 09:36:43.352418 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36,"mutex_wait_us":52}
I20260809 09:36:43.352577 16905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:36:43.348768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.131:40839 (local address 127.20.125.130:47751)
0809 09:36:43.349261 (+ 493us) negotiation.cc:107] Waiting for socket to connect
0809 09:36:43.349275 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 09:36:43.349392 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:36:43.349750 (+ 358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:36:43.349753 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:36:43.349772 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 09:36:43.349882 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:36:43.349887 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:36:43.350677 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:36:43.350679 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:36:43.351464 (+ 785us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:36:43.351473 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:36:43.352158 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:36:43.352165 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:36:43.352249 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:36:43.352266 (+ 17us) client_negotiation.cc:769] Sending connection context
0809 09:36:43.352355 (+ 89us) client_negotiation.cc:240] Negotiation successful
0809 09:36:43.352425 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":431,"mutex_wait_us":69,"thread_start_us":128,"threads_started":1}
I20260809 09:36:43.353281 16424 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fca6c797a6e44a2db7b4003db8f27512" candidate_uuid: "200658f84e7d45b699327eb83c2159cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6afb3dca1aed41e0b3a4392f7fa091f9" is_pre_election: true
I20260809 09:36:43.353391 16804 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fca6c797a6e44a2db7b4003db8f27512" candidate_uuid: "200658f84e7d45b699327eb83c2159cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "111d8fe1f13d42eaabc559d89832bc6d" is_pre_election: true
I20260809 09:36:43.353626 16424 raft_consensus.cc:2454] T fca6c797a6e44a2db7b4003db8f27512 P 6afb3dca1aed41e0b3a4392f7fa091f9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 200658f84e7d45b699327eb83c2159cb in term 0.
I20260809 09:36:43.353699 16804 raft_consensus.cc:2454] T fca6c797a6e44a2db7b4003db8f27512 P 111d8fe1f13d42eaabc559d89832bc6d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 200658f84e7d45b699327eb83c2159cb in term 0.
I20260809 09:36:43.354209 16513 leader_election.cc:302] T fca6c797a6e44a2db7b4003db8f27512 P 200658f84e7d45b699327eb83c2159cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 200658f84e7d45b699327eb83c2159cb, 6afb3dca1aed41e0b3a4392f7fa091f9; no voters:
I20260809 09:36:43.354533 16903 raft_consensus.cc:2781] T fca6c797a6e44a2db7b4003db8f27512 P 200658f84e7d45b699327eb83c2159cb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 09:36:43.354671 16903 raft_consensus.cc:482] T fca6c797a6e44a2db7b4003db8f27512 P 200658f84e7d45b699327eb83c2159cb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 09:36:43.354708 16903 raft_consensus.cc:3037] T fca6c797a6e44a2db7b4003db8f27512 P 200658f84e7d45b699327eb83c2159cb [term 0 FOLLOWER]: Advancing to term 1
I20260809 09:36:43.355755 16903 raft_consensus.cc:504] T fca6c797a6e44a2db7b4003db8f27512 P 200658f84e7d45b699327eb83c2159cb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6afb3dca1aed41e0b3a4392f7fa091f9" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 34497 } } peers { permanent_uuid: "200658f84e7d45b699327eb83c2159cb" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 39855 } } peers { permanent_uuid: "111d8fe1f13d42eaabc559d89832bc6d" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 40839 } }
I20260809 09:36:43.356082 16903 leader_election.cc:288] T fca6c797a6e44a2db7b4003db8f27512 P 200658f84e7d45b699327eb83c2159cb [CANDIDATE]: Term 1 election: Requested vote from peers 6afb3dca1aed41e0b3a4392f7fa091f9 (127.20.125.129:34497), 111d8fe1f13d42eaabc559d89832bc6d (127.20.125.131:40839)
I20260809 09:36:43.356364 16424 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fca6c797a6e44a2db7b4003db8f27512" candidate_uuid: "200658f84e7d45b699327eb83c2159cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6afb3dca1aed41e0b3a4392f7fa091f9"
I20260809 09:36:43.356469 16804 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fca6c797a6e44a2db7b4003db8f27512" candidate_uuid: "200658f84e7d45b699327eb83c2159cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "111d8fe1f13d42eaabc559d89832bc6d"
I20260809 09:36:43.356504 16424 raft_consensus.cc:3037] T fca6c797a6e44a2db7b4003db8f27512 P 6afb3dca1aed41e0b3a4392f7fa091f9 [term 0 FOLLOWER]: Advancing to term 1
I20260809 09:36:43.356632 16804 raft_consensus.cc:3037] T fca6c797a6e44a2db7b4003db8f27512 P 111d8fe1f13d42eaabc559d89832bc6d [term 0 FOLLOWER]: Advancing to term 1
I20260809 09:36:43.357640 16804 raft_consensus.cc:2454] T fca6c797a6e44a2db7b4003db8f27512 P 111d8fe1f13d42eaabc559d89832bc6d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 200658f84e7d45b699327eb83c2159cb in term 1.
I20260809 09:36:43.357688 16424 raft_consensus.cc:2454] T fca6c797a6e44a2db7b4003db8f27512 P 6afb3dca1aed41e0b3a4392f7fa091f9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 200658f84e7d45b699327eb83c2159cb in term 1.
I20260809 09:36:43.357949 16516 leader_election.cc:302] T fca6c797a6e44a2db7b4003db8f27512 P 200658f84e7d45b699327eb83c2159cb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 111d8fe1f13d42eaabc559d89832bc6d, 200658f84e7d45b699327eb83c2159cb; no voters:
I20260809 09:36:43.358136 16903 raft_consensus.cc:2781] T fca6c797a6e44a2db7b4003db8f27512 P 200658f84e7d45b699327eb83c2159cb [term 1 FOLLOWER]: Leader election won for term 1
I20260809 09:36:43.358383 16903 raft_consensus.cc:686] T fca6c797a6e44a2db7b4003db8f27512 P 200658f84e7d45b699327eb83c2159cb [term 1 LEADER]: Becoming Leader. State: Replica: 200658f84e7d45b699327eb83c2159cb, State: Running, Role: LEADER
I20260809 09:36:43.358763 16903 consensus_queue.cc:237] T fca6c797a6e44a2db7b4003db8f27512 P 200658f84e7d45b699327eb83c2159cb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6afb3dca1aed41e0b3a4392f7fa091f9" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 34497 } } peers { permanent_uuid: "200658f84e7d45b699327eb83c2159cb" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 39855 } } peers { permanent_uuid: "111d8fe1f13d42eaabc559d89832bc6d" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 40839 } }
I20260809 09:36:43.362088 16202 catalog_manager.cc:5495] T fca6c797a6e44a2db7b4003db8f27512 P 200658f84e7d45b699327eb83c2159cb reported cstate change: term changed from 0 to 1, leader changed from <none> to 200658f84e7d45b699327eb83c2159cb (127.20.125.130). New cstate: current_term: 1 leader_uuid: "200658f84e7d45b699327eb83c2159cb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6afb3dca1aed41e0b3a4392f7fa091f9" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 34497 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "200658f84e7d45b699327eb83c2159cb" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 39855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "111d8fe1f13d42eaabc559d89832bc6d" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 40839 } health_report { overall_health: UNKNOWN } } }
I20260809 09:36:43.398371 16892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:36:43.394699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38010 (local address 127.20.125.130:39855)
0809 09:36:43.394862 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0809 09:36:43.394868 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 09:36:43.394886 (+ 18us) server_negotiation.cc:407] Connection header received
0809 09:36:43.394969 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:36:43.394971 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:36:43.395026 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:36:43.395141 (+ 115us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 09:36:43.395417 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:36:43.395924 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:36:43.396810 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:36:43.397039 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:36:43.397155 (+ 116us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:36:43.397484 (+ 329us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 09:36:43.397896 (+ 412us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 09:36:43.398057 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0809 09:36:43.398161 (+ 104us) server_negotiation.cc:299] Negotiation successful
0809 09:36:43.398196 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260809 09:36:43.404296 16804 raft_consensus.cc:1261] T fca6c797a6e44a2db7b4003db8f27512 P 111d8fe1f13d42eaabc559d89832bc6d [term 1 FOLLOWER]: Refusing update from remote peer 200658f84e7d45b699327eb83c2159cb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 09:36:43.404296 16424 raft_consensus.cc:1261] T fca6c797a6e44a2db7b4003db8f27512 P 6afb3dca1aed41e0b3a4392f7fa091f9 [term 1 FOLLOWER]: Refusing update from remote peer 200658f84e7d45b699327eb83c2159cb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 09:36:43.405035 16906 consensus_queue.cc:1035] T fca6c797a6e44a2db7b4003db8f27512 P 200658f84e7d45b699327eb83c2159cb [LEADER]: Connected to new peer: Peer: permanent_uuid: "111d8fe1f13d42eaabc559d89832bc6d" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 40839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 09:36:43.405252 16903 consensus_queue.cc:1035] T fca6c797a6e44a2db7b4003db8f27512 P 200658f84e7d45b699327eb83c2159cb [LEADER]: Connected to new peer: Peer: permanent_uuid: "6afb3dca1aed41e0b3a4392f7fa091f9" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 34497 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 09:36:43.421258 16201 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:40378:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 09:36:43.421415 16201 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:40378:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 09:36:43.423945 16201 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 929bf85b5e594d22bbcc292dd6267d50: Sending DeleteTablet for 3 replicas of tablet fca6c797a6e44a2db7b4003db8f27512
I20260809 09:36:43.424762 16578 tablet_service.cc:1514] Processing DeleteTablet for tablet fca6c797a6e44a2db7b4003db8f27512 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 09:36:43 UTC) from {username='slave'} at 127.0.0.1:52134
I20260809 09:36:43.424787 16394 tablet_service.cc:1514] Processing DeleteTablet for tablet fca6c797a6e44a2db7b4003db8f27512 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 09:36:43 UTC) from {username='slave'} at 127.0.0.1:45566
I20260809 09:36:43.424942 16777 tablet_service.cc:1514] Processing DeleteTablet for tablet fca6c797a6e44a2db7b4003db8f27512 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 09:36:43 UTC) from {username='slave'} at 127.0.0.1:39900
I20260809 09:36:43.425390 16929 tablet_replica.cc:330] T fca6c797a6e44a2db7b4003db8f27512 P 111d8fe1f13d42eaabc559d89832bc6d: stopping tablet replica
I20260809 09:36:43.425733 20982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu with pid 16293
I20260809 09:36:43.425765 16929 raft_consensus.cc:2229] T fca6c797a6e44a2db7b4003db8f27512 P 111d8fe1f13d42eaabc559d89832bc6d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 09:36:43.426071 16929 raft_consensus.cc:2258] T fca6c797a6e44a2db7b4003db8f27512 P 111d8fe1f13d42eaabc559d89832bc6d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 09:36:43.430670 16929 ts_tablet_manager.cc:1905] T fca6c797a6e44a2db7b4003db8f27512 P 111d8fe1f13d42eaabc559d89832bc6d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 09:36:43.434635 16926 tablet_replica.cc:330] T fca6c797a6e44a2db7b4003db8f27512 P 200658f84e7d45b699327eb83c2159cb: stopping tablet replica
I20260809 09:36:43.434911 16929 ts_tablet_manager.cc:1918] T fca6c797a6e44a2db7b4003db8f27512 P 111d8fe1f13d42eaabc559d89832bc6d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 09:36:43.434938 16926 raft_consensus.cc:2229] T fca6c797a6e44a2db7b4003db8f27512 P 200658f84e7d45b699327eb83c2159cb [term 1 LEADER]: Raft consensus shutting down.
I20260809 09:36:43.435037 16929 log.cc:1196] T fca6c797a6e44a2db7b4003db8f27512 P 111d8fe1f13d42eaabc559d89832bc6d: Deleting WAL directory at /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786267941944493-20982-0/minicluster-data/ts-2/wal/wals/fca6c797a6e44a2db7b4003db8f27512
I20260809 09:36:43.435298 16926 raft_consensus.cc:2258] T fca6c797a6e44a2db7b4003db8f27512 P 200658f84e7d45b699327eb83c2159cb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 09:36:43.435406 16929 ts_tablet_manager.cc:1939] T fca6c797a6e44a2db7b4003db8f27512 P 111d8fe1f13d42eaabc559d89832bc6d: Deleting consensus metadata
I20260809 09:36:43.436846 16926 ts_tablet_manager.cc:1905] T fca6c797a6e44a2db7b4003db8f27512 P 200658f84e7d45b699327eb83c2159cb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 09:36:43.437302 16178 catalog_manager.cc:4841] TS 111d8fe1f13d42eaabc559d89832bc6d (127.20.125.131:40839): tablet fca6c797a6e44a2db7b4003db8f27512 (table test-table [id=21dd47c22f2d4e2087fb280caa4c1bd8]) successfully deleted
I20260809 09:36:43.439626 16926 ts_tablet_manager.cc:1918] T fca6c797a6e44a2db7b4003db8f27512 P 200658f84e7d45b699327eb83c2159cb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 09:36:43.440014 16926 log.cc:1196] T fca6c797a6e44a2db7b4003db8f27512 P 200658f84e7d45b699327eb83c2159cb: Deleting WAL directory at /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786267941944493-20982-0/minicluster-data/ts-1/wal/wals/fca6c797a6e44a2db7b4003db8f27512
I20260809 09:36:43.440412 16926 ts_tablet_manager.cc:1939] T fca6c797a6e44a2db7b4003db8f27512 P 200658f84e7d45b699327eb83c2159cb: Deleting consensus metadata
I20260809 09:36:43.441907 16177 catalog_manager.cc:4841] TS 200658f84e7d45b699327eb83c2159cb (127.20.125.130:39855): tablet fca6c797a6e44a2db7b4003db8f27512 (table test-table [id=21dd47c22f2d4e2087fb280caa4c1bd8]) successfully deleted
W20260809 09:36:43.445938 16176 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.20.125.129:34497 (error 108)
I20260809 09:36:43.446038 20982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu with pid 16494
I20260809 09:36:43.451920 16888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:36:43.451529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.129:34497 (local address 127.0.0.1:59452)
0809 09:36:43.451739 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0809 09:36:43.451816 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.129:34497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115}
W20260809 09:36:43.452086 16176 catalog_manager.cc:4584] TS 6afb3dca1aed41e0b3a4392f7fa091f9 (127.20.125.129:34497): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet fca6c797a6e44a2db7b4003db8f27512: Network error: Client connection negotiation failed: client connection to 127.20.125.129:34497: connect: Connection refused (error 111)
I20260809 09:36:43.462977 20982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu with pid 16678
I20260809 09:36:43.476274 20982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu with pid 16153
2026-08-09T09:36:43Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (1149 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786267941944493-20982-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 09:36:43 dist-test-slave-7kzw krb5kdc[16936](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 09:36:43 dist-test-slave-7kzw krb5kdc[16936](info): set up 2 sockets
Aug 09 09:36:43 dist-test-slave-7kzw krb5kdc[16936](info): commencing operation
krb5kdc: starting...
W20260809 09:36:44.802379 20982 mini_kdc.cc:121] Time spent starting KDC: real 1.302s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786267941944493-20982-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786267941944493-20982-0/krb5kdc/test-user.keytab.
Aug 09 09:36:44 dist-test-slave-7kzw krb5kdc[16936](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268204, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T09:36:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T09:36:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.125.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.125.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.125.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.190.keytab.
Entry for principal kudu/127.20.125.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.190.keytab.
I20260809 09:36:44.945924 20982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
/tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786267941944493-20982-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786267941944493-20982-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786267941944493-20982-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786267941944493-20982-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.125.190:45703
--webserver_interface=127.20.125.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.190.keytab
--principal=kudu/127.20.125.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.125.148:43459
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.125.190:45703
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786267941944493-20982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786267941944493-20982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786267941944493-20982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 09:36:45.058739 16956 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 09:36:45.058998 16956 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 09:36:45.059038 16956 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 09:36:45.063052 16956 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 09:36:45.063107 16956 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 09:36:45.063126 16956 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 09:36:45.063143 16956 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 09:36:45.063158 16956 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 09:36:45.068392 16956 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.125.148:43459
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786267941944493-20982-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786267941944493-20982-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.125.190:45703
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.190.keytab
--principal=kudu/127.20.125.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.125.190:45703
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786267941944493-20982-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.125.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16956
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786267941944493-20982-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 08:38:55 UTC on ccycloud.quasar-njjleu.root.comops.site
build id 4575935
I20260809 09:36:45.069407 16956 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 09:36:45.070428 16956 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 09:36:45.079018 16963 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:36:45.079099 16964 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:36:45.079113 16966 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:36:45.079245 16956 server_base.cc:1034] running on GCE node
I20260809 09:36:45.079984 16956 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 09:36:45.081045 16956 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 09:36:45.082240 16956 hybrid_clock.cc:648] HybridClock initialized: now 1786268205082214 us; error 37 us; skew 500 ppm
Aug 09 09:36:45 dist-test-slave-7kzw krb5kdc[16936](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268205, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 09:36:45.085242 16956 init.cc:377] Logged in from keytab as kudu/127.20.125.190@KRBTEST.COM (short username kudu)
I20260809 09:36:45.086139 16956 webserver.cc:460] Webserver started at http://127.20.125.190:39897/ using document root <none> and password file <none>
I20260809 09:36:45.086786 16956 fs_manager.cc:359] Metadata directory not provided
I20260809 09:36:45.086885 16956 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 09:36:45.087118 16956 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 09:36:45.089102 16956 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786267941944493-20982-0/minicluster-data/master-0/data/instance:
uuid: "13463be3de9b4ec9bd6d7ecf40625d98"
format_stamp: "Formatted at 2026-08-09 09:36:45 on dist-test-slave-7kzw"
server_key: "7137ba6ef14832fee18c390ff66a8ba8"
server_key_iv: "d9768ad89798a581735fb1156c27f34a"
server_key_version: "encryptionkey@0"
I20260809 09:36:45.089676 16956 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786267941944493-20982-0/minicluster-data/master-0/wal/instance:
uuid: "13463be3de9b4ec9bd6d7ecf40625d98"
format_stamp: "Formatted at 2026-08-09 09:36:45 on dist-test-slave-7kzw"
server_key: "7137ba6ef14832fee18c390ff66a8ba8"
server_key_iv: "d9768ad89798a581735fb1156c27f34a"
server_key_version: "encryptionkey@0"
I20260809 09:36:45.093381 16956 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.003s
I20260809 09:36:45.095901 16978 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 09:36:45.097153 16956 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 09:36:45.097316 16956 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786267941944493-20982-0/minicluster-data/master-0/data,/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786267941944493-20982-0/minicluster-data/master-0/wal
uuid: "13463be3de9b4ec9bd6d7ecf40625d98"
format_stamp: "Formatted at 2026-08-09 09:36:45 on dist-test-slave-7kzw"
server_key: "7137ba6ef14832fee18c390ff66a8ba8"
server_key_iv: "d9768ad89798a581735fb1156c27f34a"
server_key_version: "encryptionkey@0"
I20260809 09:36:45.097433 16956 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786267941944493-20982-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786267941944493-20982-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786267941944493-20982-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 09:36:45.121379 16956 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 09:36:45.124826 16956 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 09:36:45.125046 16956 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 09:36:45.132846 16956 rpc_server.cc:307] RPC server started. Bound to: 127.20.125.190:45703
I20260809 09:36:45.132869 17057 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.125.190:45703 every 8 connection(s)
I20260809 09:36:45.133869 16956 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786267941944493-20982-0/minicluster-data/master-0/data/info.pb
I20260809 09:36:45.136611 17058 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 09:36:45.142915 17058 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 13463be3de9b4ec9bd6d7ecf40625d98: Bootstrap starting.
I20260809 09:36:45.143271 20982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu as pid 16956
I20260809 09:36:45.143421 20982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786267941944493-20982-0/minicluster-data/master-0/wal/instance
I20260809 09:36:45.143715 20982 external_mini_cluster.cc:1428] Setting key 5b1d9044db6218d4cba61325dc40a182
I20260809 09:36:45.145479 17058 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 13463be3de9b4ec9bd6d7ecf40625d98: Neither blocks nor log segments found. Creating new log.
I20260809 09:36:45.146215 17058 log.cc:824] T 00000000000000000000000000000000 P 13463be3de9b4ec9bd6d7ecf40625d98: Log is configured to *not* fsync() on all Append() calls
I20260809 09:36:45.148124 17058 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 13463be3de9b4ec9bd6d7ecf40625d98: No bootstrap required, opened a new log
I20260809 09:36:45.150964 17058 raft_consensus.cc:348] T 00000000000000000000000000000000 P 13463be3de9b4ec9bd6d7ecf40625d98 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13463be3de9b4ec9bd6d7ecf40625d98" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 45703 } }
I20260809 09:36:45.151197 17058 raft_consensus.cc:374] T 00000000000000000000000000000000 P 13463be3de9b4ec9bd6d7ecf40625d98 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 09:36:45.151291 17058 raft_consensus.cc:729] T 00000000000000000000000000000000 P 13463be3de9b4ec9bd6d7ecf40625d98 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13463be3de9b4ec9bd6d7ecf40625d98, State: Initialized, Role: FOLLOWER
I20260809 09:36:45.151794 17058 consensus_queue.cc:260] T 00000000000000000000000000000000 P 13463be3de9b4ec9bd6d7ecf40625d98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13463be3de9b4ec9bd6d7ecf40625d98" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 45703 } }
I20260809 09:36:45.151971 17058 raft_consensus.cc:388] T 00000000000000000000000000000000 P 13463be3de9b4ec9bd6d7ecf40625d98 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 09:36:45.152046 17058 raft_consensus.cc:482] T 00000000000000000000000000000000 P 13463be3de9b4ec9bd6d7ecf40625d98 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 09:36:45.152156 17058 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 13463be3de9b4ec9bd6d7ecf40625d98 [term 0 FOLLOWER]: Advancing to term 1
I20260809 09:36:45.153244 17058 raft_consensus.cc:504] T 00000000000000000000000000000000 P 13463be3de9b4ec9bd6d7ecf40625d98 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13463be3de9b4ec9bd6d7ecf40625d98" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 45703 } }
Aug 09 09:36:45 dist-test-slave-7kzw krb5kdc[16936](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268204, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.125.190@KRBTEST.COM
I20260809 09:36:45.153575 17058 leader_election.cc:302] T 00000000000000000000000000000000 P 13463be3de9b4ec9bd6d7ecf40625d98 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 13463be3de9b4ec9bd6d7ecf40625d98; no voters:
I20260809 09:36:45.153875 17058 leader_election.cc:288] T 00000000000000000000000000000000 P 13463be3de9b4ec9bd6d7ecf40625d98 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 09:36:45.153995 17067 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 13463be3de9b4ec9bd6d7ecf40625d98 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 09:36:45.154241 17067 raft_consensus.cc:686] T 00000000000000000000000000000000 P 13463be3de9b4ec9bd6d7ecf40625d98 [term 1 LEADER]: Becoming Leader. State: Replica: 13463be3de9b4ec9bd6d7ecf40625d98, State: Running, Role: LEADER
I20260809 09:36:45.154609 17067 consensus_queue.cc:237] T 00000000000000000000000000000000 P 13463be3de9b4ec9bd6d7ecf40625d98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13463be3de9b4ec9bd6d7ecf40625d98" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 45703 } }
I20260809 09:36:45.155166 17058 sys_catalog.cc:564] T 00000000000000000000000000000000 P 13463be3de9b4ec9bd6d7ecf40625d98 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 09:36:45.155879 17067 sys_catalog.cc:455] T 00000000000000000000000000000000 P 13463be3de9b4ec9bd6d7ecf40625d98 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 13463be3de9b4ec9bd6d7ecf40625d98. Latest consensus state: current_term: 1 leader_uuid: "13463be3de9b4ec9bd6d7ecf40625d98" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13463be3de9b4ec9bd6d7ecf40625d98" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 45703 } } }
I20260809 09:36:45.155999 17067 sys_catalog.cc:458] T 00000000000000000000000000000000 P 13463be3de9b4ec9bd6d7ecf40625d98 [sys.catalog]: This master's current role is: LEADER
I20260809 09:36:45.156678 17068 sys_catalog.cc:455] T 00000000000000000000000000000000 P 13463be3de9b4ec9bd6d7ecf40625d98 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "13463be3de9b4ec9bd6d7ecf40625d98" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13463be3de9b4ec9bd6d7ecf40625d98" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 45703 } } }
I20260809 09:36:45.156795 17068 sys_catalog.cc:458] T 00000000000000000000000000000000 P 13463be3de9b4ec9bd6d7ecf40625d98 [sys.catalog]: This master's current role is: LEADER
I20260809 09:36:45.159895 17065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:36:45.145077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48040 (local address 127.20.125.190:45703)
0809 09:36:45.145504 (+ 427us) server_negotiation.cc:206] Beginning negotiation
0809 09:36:45.145525 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0809 09:36:45.145608 (+ 83us) server_negotiation.cc:407] Connection header received
0809 09:36:45.146286 (+ 678us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:36:45.146302 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:36:45.146738 (+ 436us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:36:45.147059 (+ 321us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:36:45.147887 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:36:45.148930 (+ 1043us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:36:45.149803 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:36:45.150867 (+ 1064us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:36:45.151053 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:36:45.154078 (+ 3025us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:36:45.154085 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:36:45.154101 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:36:45.154137 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:36:45.156557 (+ 2420us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:36:45.157244 (+ 687us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:36:45.157249 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:36:45.157254 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:36:45.157408 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:36:45.157771 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:36:45.157774 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:36:45.157776 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:36:45.158147 (+ 371us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:36:45.158381 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0809 09:36:45.158708 (+ 327us) server_negotiation.cc:299] Negotiation successful
0809 09:36:45.158995 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":271,"thread_start_us":149,"threads_started":1}
W20260809 09:36:45.160154 17085 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 13463be3de9b4ec9bd6d7ecf40625d98: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 09:36:45.160236 17085 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 09:36:45.160313 17078 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 09:36:45.161027 17078 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 09:36:45.165864 17078 catalog_manager.cc:1347] Generated new cluster ID: c158075b8ce943569b8e9f828ca8d3d7
I20260809 09:36:45.165928 17078 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 09:36:45.185458 17078 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 09:36:45.186383 17078 catalog_manager.cc:1504] Loading token signing keys...
I20260809 09:36:45.200659 17078 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 13463be3de9b4ec9bd6d7ecf40625d98: Generated new TSK 0
I20260809 09:36:45.201512 17078 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.20.125.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.125.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.125.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.129.keytab.
Entry for principal kudu/127.20.125.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.129.keytab.
I20260809 09:36:45.241667 20982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
/tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786267941944493-20982-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786267941944493-20982-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786267941944493-20982-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786267941944493-20982-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.125.129:0
--local_ip_for_outbound_sockets=127.20.125.129
--webserver_interface=127.20.125.129
--webserver_port=0
--tserver_master_addrs=127.20.125.190:45703
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.129.keytab
--principal=kudu/127.20.125.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.125.148:43459
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786267941944493-20982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786267941944493-20982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786267941944493-20982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 09:36:45.354969 17094 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 09:36:45.355252 17094 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 09:36:45.355294 17094 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 09:36:45.359591 17094 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 09:36:45.359647 17094 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 09:36:45.359702 17094 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.125.129
I20260809 09:36:45.364917 17094 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.125.148:43459
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786267941944493-20982-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786267941944493-20982-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.129.keytab
--principal=kudu/127.20.125.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.125.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786267941944493-20982-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.125.129
--webserver_port=0
--tserver_master_addrs=127.20.125.190:45703
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17094
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.125.129
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786267941944493-20982-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 08:38:55 UTC on ccycloud.quasar-njjleu.root.comops.site
build id 4575935
I20260809 09:36:45.365978 17094 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 09:36:45.367063 17094 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 09:36:45.376912 17094 server_base.cc:1034] running on GCE node
W20260809 09:36:45.376875 17109 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:36:45.376875 17113 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:36:45.377108 17106 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:36:45.377574 17094 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 09:36:45.378198 17094 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 09:36:45.379367 17094 hybrid_clock.cc:648] HybridClock initialized: now 1786268205379346 us; error 32 us; skew 500 ppm
Aug 09 09:36:45 dist-test-slave-7kzw krb5kdc[16936](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268205, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 09:36:45.382122 17094 init.cc:377] Logged in from keytab as kudu/127.20.125.129@KRBTEST.COM (short username kudu)
I20260809 09:36:45.383008 17094 webserver.cc:460] Webserver started at http://127.20.125.129:41051/ using document root <none> and password file <none>
I20260809 09:36:45.383550 17094 fs_manager.cc:359] Metadata directory not provided
I20260809 09:36:45.383610 17094 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 09:36:45.383896 17094 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 09:36:45.385842 17094 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786267941944493-20982-0/minicluster-data/ts-0/data/instance:
uuid: "d65bb7049acb48a29e27fc30a4cdc1aa"
format_stamp: "Formatted at 2026-08-09 09:36:45 on dist-test-slave-7kzw"
server_key: "0ea38d207089f28fcc7881272f56a3ad"
server_key_iv: "17dba1ea9e0aab576c4a611e640ce5f9"
server_key_version: "encryptionkey@0"
I20260809 09:36:45.386404 17094 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786267941944493-20982-0/minicluster-data/ts-0/wal/instance:
uuid: "d65bb7049acb48a29e27fc30a4cdc1aa"
format_stamp: "Formatted at 2026-08-09 09:36:45 on dist-test-slave-7kzw"
server_key: "0ea38d207089f28fcc7881272f56a3ad"
server_key_iv: "17dba1ea9e0aab576c4a611e640ce5f9"
server_key_version: "encryptionkey@0"
I20260809 09:36:45.390057 17094 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260809 09:36:45.392535 17124 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 09:36:45.393877 17094 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 09:36:45.394049 17094 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786267941944493-20982-0/minicluster-data/ts-0/data,/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786267941944493-20982-0/minicluster-data/ts-0/wal
uuid: "d65bb7049acb48a29e27fc30a4cdc1aa"
format_stamp: "Formatted at 2026-08-09 09:36:45 on dist-test-slave-7kzw"
server_key: "0ea38d207089f28fcc7881272f56a3ad"
server_key_iv: "17dba1ea9e0aab576c4a611e640ce5f9"
server_key_version: "encryptionkey@0"
I20260809 09:36:45.394181 17094 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786267941944493-20982-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786267941944493-20982-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786267941944493-20982-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 09:36:45.410193 17094 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 09:36:45.413353 17094 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 09:36:45.413559 17094 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 09:36:45.414142 17094 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 09:36:45.415007 17094 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 09:36:45.415078 17094 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 09:36:45.415151 17094 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 09:36:45.415203 17094 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 09:36:45.423753 17094 rpc_server.cc:307] RPC server started. Bound to: 127.20.125.129:36585
I20260809 09:36:45.423795 17284 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.125.129:36585 every 8 connection(s)
I20260809 09:36:45.424593 17094 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786267941944493-20982-0/minicluster-data/ts-0/data/info.pb
I20260809 09:36:45.428812 20982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu as pid 17094
I20260809 09:36:45.428962 20982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786267941944493-20982-0/minicluster-data/ts-0/wal/instance
I20260809 09:36:45.429268 20982 external_mini_cluster.cc:1428] Setting key 2489a70a5aa3d8a5e652ab0d057c8987
Aug 09 09:36:45 dist-test-slave-7kzw krb5kdc[16936](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268205, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.129@KRBTEST.COM for kudu/127.20.125.190@KRBTEST.COM
I20260809 09:36:45.438721 17065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:36:45.426402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.129:58601 (local address 127.20.125.190:45703)
0809 09:36:45.426566 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0809 09:36:45.426570 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 09:36:45.427433 (+ 863us) server_negotiation.cc:407] Connection header received
0809 09:36:45.428864 (+ 1431us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:36:45.428867 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:36:45.428927 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:36:45.429028 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:36:45.430195 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:36:45.430790 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:36:45.431912 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:36:45.432144 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:36:45.432256 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:36:45.435019 (+ 2763us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:36:45.435022 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:36:45.435024 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:36:45.435047 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:36:45.436551 (+ 1504us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:36:45.437095 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:36:45.437100 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:36:45.437101 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:36:45.437197 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:36:45.437558 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:36:45.437564 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:36:45.437565 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:36:45.437755 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:36:45.437865 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0809 09:36:45.438366 (+ 501us) server_negotiation.cc:299] Negotiation successful
0809 09:36:45.438538 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260809 09:36:45.439505 17295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:36:45.426824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:45703 (local address 127.20.125.129:58601)
0809 09:36:45.427269 (+ 445us) negotiation.cc:107] Waiting for socket to connect
0809 09:36:45.427328 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0809 09:36:45.428642 (+ 1314us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:36:45.429212 (+ 570us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:36:45.429219 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:36:45.429601 (+ 382us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:36:45.430050 (+ 449us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:36:45.430062 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:36:45.430920 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:36:45.430923 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:36:45.431782 (+ 859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:36:45.431791 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:36:45.432294 (+ 503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:36:45.432298 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:36:45.432468 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:36:45.433085 (+ 617us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 09:36:45.433110 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:36:45.434845 (+ 1735us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 09:36:45.436681 (+ 1836us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:36:45.436687 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:36:45.436711 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:36:45.436988 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:36:45.437301 (+ 313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:36:45.437304 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:36:45.437306 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:36:45.437456 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:36:45.437866 (+ 410us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 09:36:45.437874 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 09:36:45.438128 (+ 254us) client_negotiation.cc:769] Sending connection context
0809 09:36:45.438322 (+ 194us) client_negotiation.cc:240] Negotiation successful
0809 09:36:45.438673 (+ 351us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":256,"thread_start_us":104,"threads_started":1}
WARNING: no policy specified for kudu/127.20.125.130@KRBTEST.COM; defaulting to no policy
I20260809 09:36:45.440773 17291 heartbeater.cc:347] Connected to a master server at 127.20.125.190:45703
I20260809 09:36:45.441031 17291 heartbeater.cc:464] Registering TS with master...
I20260809 09:36:45.441545 17291 heartbeater.cc:511] Master 127.20.125.190:45703 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.125.130@KRBTEST.COM" created.
I20260809 09:36:45.443370 17001 ts_manager.cc:194] Registered new tserver with Master: d65bb7049acb48a29e27fc30a4cdc1aa (127.20.125.129:36585)
I20260809 09:36:45.444574 17001 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.125.129@KRBTEST.COM'} at 127.20.125.129:58601
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.125.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.130.keytab.
Entry for principal kudu/127.20.125.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.130.keytab.
I20260809 09:36:45.459479 20982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
/tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786267941944493-20982-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786267941944493-20982-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786267941944493-20982-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786267941944493-20982-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.125.130:0
--local_ip_for_outbound_sockets=127.20.125.130
--webserver_interface=127.20.125.130
--webserver_port=0
--tserver_master_addrs=127.20.125.190:45703
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.130.keytab
--principal=kudu/127.20.125.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.125.148:43459
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786267941944493-20982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786267941944493-20982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786267941944493-20982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 09:36:45.573979 17298 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 09:36:45.574227 17298 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 09:36:45.574270 17298 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 09:36:45.578219 17298 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 09:36:45.578276 17298 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 09:36:45.578333 17298 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.125.130
I20260809 09:36:45.583559 17298 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.125.148:43459
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786267941944493-20982-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786267941944493-20982-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.130.keytab
--principal=kudu/127.20.125.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.125.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786267941944493-20982-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.125.130
--webserver_port=0
--tserver_master_addrs=127.20.125.190:45703
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17298
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.125.130
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786267941944493-20982-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 08:38:55 UTC on ccycloud.quasar-njjleu.root.comops.site
build id 4575935
I20260809 09:36:45.584512 17298 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 09:36:45.585482 17298 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 09:36:45.595073 17309 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:36:45.595165 17305 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:36:45.595084 17307 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:36:45.595381 17298 server_base.cc:1034] running on GCE node
I20260809 09:36:45.595945 17298 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 09:36:45.596593 17298 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 09:36:45.597860 17298 hybrid_clock.cc:648] HybridClock initialized: now 1786268205597825 us; error 49 us; skew 500 ppm
Aug 09 09:36:45 dist-test-slave-7kzw krb5kdc[16936](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268205, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 09:36:45.601076 17298 init.cc:377] Logged in from keytab as kudu/127.20.125.130@KRBTEST.COM (short username kudu)
I20260809 09:36:45.601990 17298 webserver.cc:460] Webserver started at http://127.20.125.130:39101/ using document root <none> and password file <none>
I20260809 09:36:45.602653 17298 fs_manager.cc:359] Metadata directory not provided
I20260809 09:36:45.602757 17298 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 09:36:45.603070 17298 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 09:36:45.605207 17298 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786267941944493-20982-0/minicluster-data/ts-1/data/instance:
uuid: "e578eee054a646149aa53b8fd83c960a"
format_stamp: "Formatted at 2026-08-09 09:36:45 on dist-test-slave-7kzw"
server_key: "5e3c82f40e74a604e498a008e5c9450d"
server_key_iv: "19f32794c5c9f02c9f5682dc5965c25a"
server_key_version: "encryptionkey@0"
I20260809 09:36:45.605798 17298 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786267941944493-20982-0/minicluster-data/ts-1/wal/instance:
uuid: "e578eee054a646149aa53b8fd83c960a"
format_stamp: "Formatted at 2026-08-09 09:36:45 on dist-test-slave-7kzw"
server_key: "5e3c82f40e74a604e498a008e5c9450d"
server_key_iv: "19f32794c5c9f02c9f5682dc5965c25a"
server_key_version: "encryptionkey@0"
I20260809 09:36:45.609678 17298 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260809 09:36:45.612288 17316 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 09:36:45.613529 17298 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 09:36:45.613684 17298 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786267941944493-20982-0/minicluster-data/ts-1/data,/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786267941944493-20982-0/minicluster-data/ts-1/wal
uuid: "e578eee054a646149aa53b8fd83c960a"
format_stamp: "Formatted at 2026-08-09 09:36:45 on dist-test-slave-7kzw"
server_key: "5e3c82f40e74a604e498a008e5c9450d"
server_key_iv: "19f32794c5c9f02c9f5682dc5965c25a"
server_key_version: "encryptionkey@0"
I20260809 09:36:45.613808 17298 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786267941944493-20982-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786267941944493-20982-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786267941944493-20982-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 09:36:45.634009 17298 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 09:36:45.637436 17298 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 09:36:45.637676 17298 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 09:36:45.638339 17298 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 09:36:45.639271 17298 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 09:36:45.639344 17298 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 09:36:45.639413 17298 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 09:36:45.639461 17298 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 09:36:45.648538 17298 rpc_server.cc:307] RPC server started. Bound to: 127.20.125.130:40551
I20260809 09:36:45.648567 17490 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.125.130:40551 every 8 connection(s)
I20260809 09:36:45.649452 17298 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786267941944493-20982-0/minicluster-data/ts-1/data/info.pb
I20260809 09:36:45.656536 20982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu as pid 17298
I20260809 09:36:45.656697 20982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786267941944493-20982-0/minicluster-data/ts-1/wal/instance
I20260809 09:36:45.657030 20982 external_mini_cluster.cc:1428] Setting key 7416a8de245e8c2eceb28a22cfe36f27
Aug 09 09:36:45 dist-test-slave-7kzw krb5kdc[16936](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268205, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.130@KRBTEST.COM for kudu/127.20.125.190@KRBTEST.COM
I20260809 09:36:45.665483 17065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:36:45.651388 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.130:53569 (local address 127.20.125.190:45703)
0809 09:36:45.651533 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0809 09:36:45.651537 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 09:36:45.652374 (+ 837us) server_negotiation.cc:407] Connection header received
0809 09:36:45.653670 (+ 1296us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:36:45.653674 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:36:45.653722 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:36:45.653806 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:36:45.655073 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:36:45.655659 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:36:45.657116 (+ 1457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:36:45.657360 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:36:45.657474 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:36:45.660808 (+ 3334us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:36:45.660814 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:36:45.660818 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:36:45.660852 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:36:45.662816 (+ 1964us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:36:45.663498 (+ 682us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:36:45.663503 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:36:45.663507 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:36:45.663615 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:36:45.664062 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:36:45.664067 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:36:45.664069 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:36:45.664350 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:36:45.664465 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0809 09:36:45.665177 (+ 712us) server_negotiation.cc:299] Negotiation successful
0809 09:36:45.665300 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260809 09:36:45.666723 17493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:36:45.651755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:45703 (local address 127.20.125.130:53569)
0809 09:36:45.652216 (+ 461us) negotiation.cc:107] Waiting for socket to connect
0809 09:36:45.652260 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 09:36:45.653467 (+ 1207us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:36:45.653969 (+ 502us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:36:45.653976 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:36:45.654340 (+ 364us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:36:45.654939 (+ 599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:36:45.654954 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:36:45.655769 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:36:45.655773 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:36:45.656999 (+ 1226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:36:45.657012 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:36:45.657730 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:36:45.657733 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:36:45.657870 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:36:45.658654 (+ 784us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 09:36:45.658670 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:36:45.660611 (+ 1941us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 09:36:45.662964 (+ 2353us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:36:45.662974 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:36:45.662992 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:36:45.663377 (+ 385us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:36:45.663730 (+ 353us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:36:45.663735 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:36:45.663740 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:36:45.663946 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:36:45.664488 (+ 542us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 09:36:45.664498 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 09:36:45.664859 (+ 361us) client_negotiation.cc:769] Sending connection context
0809 09:36:45.665095 (+ 236us) client_negotiation.cc:240] Negotiation successful
0809 09:36:45.665469 (+ 374us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":113,"threads_started":1}
I20260809 09:36:45.668143 17491 heartbeater.cc:347] Connected to a master server at 127.20.125.190:45703
I20260809 09:36:45.668392 17491 heartbeater.cc:464] Registering TS with master...
I20260809 09:36:45.668946 17491 heartbeater.cc:511] Master 127.20.125.190:45703 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.20.125.131@KRBTEST.COM; defaulting to no policy
I20260809 09:36:45.670107 17001 ts_manager.cc:194] Registered new tserver with Master: e578eee054a646149aa53b8fd83c960a (127.20.125.130:40551)
I20260809 09:36:45.670899 17001 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.125.130@KRBTEST.COM'} at 127.20.125.130:53569
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.125.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.125.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.131.keytab.
Entry for principal kudu/127.20.125.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.131.keytab.
I20260809 09:36:45.690029 20982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
/tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786267941944493-20982-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786267941944493-20982-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786267941944493-20982-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786267941944493-20982-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.125.131:0
--local_ip_for_outbound_sockets=127.20.125.131
--webserver_interface=127.20.125.131
--webserver_port=0
--tserver_master_addrs=127.20.125.190:45703
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.131.keytab
--principal=kudu/127.20.125.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.125.148:43459
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786267941944493-20982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786267941944493-20982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786267941944493-20982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 09:36:45.808348 17496 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 09:36:45.808614 17496 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 09:36:45.808668 17496 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 09:36:45.812986 17496 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 09:36:45.813036 17496 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 09:36:45.813091 17496 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.125.131
I20260809 09:36:45.818388 17496 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.125.148:43459
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786267941944493-20982-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786267941944493-20982-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.131.keytab
--principal=kudu/127.20.125.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.125.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786267941944493-20982-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.125.131
--webserver_port=0
--tserver_master_addrs=127.20.125.190:45703
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17496
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.125.131
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786267941944493-20982-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 08:38:55 UTC on ccycloud.quasar-njjleu.root.comops.site
build id 4575935
I20260809 09:36:45.819386 17496 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 09:36:45.820343 17496 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 09:36:45.830102 17504 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:36:45.830140 17503 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:36:45.830156 17496 server_base.cc:1034] running on GCE node
W20260809 09:36:45.830164 17506 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:36:45.830842 17496 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 09:36:45.831579 17496 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 09:36:45.832763 17496 hybrid_clock.cc:648] HybridClock initialized: now 1786268205832748 us; error 25 us; skew 500 ppm
Aug 09 09:36:45 dist-test-slave-7kzw krb5kdc[16936](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268205, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 09:36:45.835861 17496 init.cc:377] Logged in from keytab as kudu/127.20.125.131@KRBTEST.COM (short username kudu)
I20260809 09:36:45.836812 17496 webserver.cc:460] Webserver started at http://127.20.125.131:42005/ using document root <none> and password file <none>
I20260809 09:36:45.837438 17496 fs_manager.cc:359] Metadata directory not provided
I20260809 09:36:45.837569 17496 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 09:36:45.837831 17496 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 09:36:45.840108 17496 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786267941944493-20982-0/minicluster-data/ts-2/data/instance:
uuid: "25f66191e68f4e6fb6c82b73bbb0306d"
format_stamp: "Formatted at 2026-08-09 09:36:45 on dist-test-slave-7kzw"
server_key: "0a85038c1e2a1017bab09852e05af609"
server_key_iv: "c6b207e9034b52590640853c14bbc580"
server_key_version: "encryptionkey@0"
I20260809 09:36:45.840756 17496 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786267941944493-20982-0/minicluster-data/ts-2/wal/instance:
uuid: "25f66191e68f4e6fb6c82b73bbb0306d"
format_stamp: "Formatted at 2026-08-09 09:36:45 on dist-test-slave-7kzw"
server_key: "0a85038c1e2a1017bab09852e05af609"
server_key_iv: "c6b207e9034b52590640853c14bbc580"
server_key_version: "encryptionkey@0"
I20260809 09:36:45.844732 17496 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 09:36:45.847283 17517 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 09:36:45.848477 17496 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 09:36:45.848639 17496 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786267941944493-20982-0/minicluster-data/ts-2/data,/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786267941944493-20982-0/minicluster-data/ts-2/wal
uuid: "25f66191e68f4e6fb6c82b73bbb0306d"
format_stamp: "Formatted at 2026-08-09 09:36:45 on dist-test-slave-7kzw"
server_key: "0a85038c1e2a1017bab09852e05af609"
server_key_iv: "c6b207e9034b52590640853c14bbc580"
server_key_version: "encryptionkey@0"
I20260809 09:36:45.848753 17496 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786267941944493-20982-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786267941944493-20982-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786267941944493-20982-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 09:36:45.861135 17496 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 09:36:45.863911 17496 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 09:36:45.864120 17496 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 09:36:45.864684 17496 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 09:36:45.865491 17496 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 09:36:45.865566 17496 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 09:36:45.865633 17496 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 09:36:45.865679 17496 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 09:36:45.874565 17496 rpc_server.cc:307] RPC server started. Bound to: 127.20.125.131:38749
I20260809 09:36:45.874614 17687 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.125.131:38749 every 8 connection(s)
I20260809 09:36:45.875445 17496 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786267941944493-20982-0/minicluster-data/ts-2/data/info.pb
I20260809 09:36:45.877007 20982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu as pid 17496
I20260809 09:36:45.877152 20982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786267941944493-20982-0/minicluster-data/ts-2/wal/instance
I20260809 09:36:45.877499 20982 external_mini_cluster.cc:1428] Setting key 20af29a634003a3d909ab278ca70dc23
Aug 09 09:36:45 dist-test-slave-7kzw krb5kdc[16936](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268205, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.131@KRBTEST.COM for kudu/127.20.125.190@KRBTEST.COM
I20260809 09:36:45.890637 17065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:36:45.877460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.131:59893 (local address 127.20.125.190:45703)
0809 09:36:45.877611 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0809 09:36:45.877617 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 09:36:45.878589 (+ 972us) server_negotiation.cc:407] Connection header received
0809 09:36:45.880355 (+ 1766us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:36:45.880357 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:36:45.880405 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:36:45.880478 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:36:45.881598 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:36:45.882117 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:36:45.883272 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:36:45.883481 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:36:45.883565 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:36:45.886290 (+ 2725us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:36:45.886293 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:36:45.886295 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:36:45.886319 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:36:45.888043 (+ 1724us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:36:45.888687 (+ 644us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:36:45.888693 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:36:45.888696 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:36:45.888802 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:36:45.889256 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:36:45.889262 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:36:45.889265 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:36:45.889483 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:36:45.889625 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0809 09:36:45.890244 (+ 619us) server_negotiation.cc:299] Negotiation successful
0809 09:36:45.890384 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260809 09:36:45.891657 17690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:36:45.877839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:45703 (local address 127.20.125.131:59893)
0809 09:36:45.878414 (+ 575us) negotiation.cc:107] Waiting for socket to connect
0809 09:36:45.878471 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0809 09:36:45.880141 (+ 1670us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:36:45.880634 (+ 493us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:36:45.880640 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:36:45.880975 (+ 335us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:36:45.881456 (+ 481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:36:45.881469 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:36:45.882221 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:36:45.882223 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:36:45.883163 (+ 940us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:36:45.883172 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:36:45.883646 (+ 474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:36:45.883650 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:36:45.883815 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:36:45.884471 (+ 656us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 09:36:45.884487 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:36:45.886117 (+ 1630us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 09:36:45.888203 (+ 2086us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:36:45.888210 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:36:45.888225 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:36:45.888554 (+ 329us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:36:45.888928 (+ 374us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:36:45.888934 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:36:45.888937 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:36:45.889132 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:36:45.889635 (+ 503us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 09:36:45.889645 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 09:36:45.889922 (+ 277us) client_negotiation.cc:769] Sending connection context
0809 09:36:45.890145 (+ 223us) client_negotiation.cc:240] Negotiation successful
0809 09:36:45.890528 (+ 383us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":321,"thread_start_us":157,"threads_started":1}
I20260809 09:36:45.892807 17688 heartbeater.cc:347] Connected to a master server at 127.20.125.190:45703
I20260809 09:36:45.893101 17688 heartbeater.cc:464] Registering TS with master...
I20260809 09:36:45.893617 17688 heartbeater.cc:511] Master 127.20.125.190:45703 requested a full tablet report, sending...
I20260809 09:36:45.894804 17001 ts_manager.cc:194] Registered new tserver with Master: 25f66191e68f4e6fb6c82b73bbb0306d (127.20.125.131:38749)
I20260809 09:36:45.895375 17001 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.125.131@KRBTEST.COM'} at 127.20.125.131:59893
I20260809 09:36:45.903985 20982 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 09:36:45.914713 17065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:36:45.906324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48044 (local address 127.20.125.190:45703)
0809 09:36:45.906514 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0809 09:36:45.906520 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 09:36:45.906770 (+ 250us) server_negotiation.cc:407] Connection header received
0809 09:36:45.906950 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:36:45.906953 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:36:45.907007 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:36:45.907099 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:36:45.907922 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:36:45.908489 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:36:45.909432 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:36:45.909674 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:36:45.909788 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:36:45.911395 (+ 1607us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:36:45.911399 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:36:45.911402 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:36:45.911430 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:36:45.912926 (+ 1496us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:36:45.913393 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:36:45.913397 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:36:45.913399 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:36:45.913487 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:36:45.913774 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:36:45.913778 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:36:45.913780 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:36:45.914005 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:36:45.914156 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0809 09:36:45.914391 (+ 235us) server_negotiation.cc:299] Negotiation successful
0809 09:36:45.914533 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260809 09:36:45.917647 17001 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48044:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 09:36:45.921072 17001 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 09:36:45.936398 17702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:36:45.930274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.129:36585 (local address 127.0.0.1:55170)
0809 09:36:45.930970 (+ 696us) negotiation.cc:107] Waiting for socket to connect
0809 09:36:45.931022 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0809 09:36:45.931148 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:36:45.931985 (+ 837us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:36:45.931991 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:36:45.932017 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 09:36:45.932184 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:36:45.932192 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:36:45.933538 (+ 1346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:36:45.933542 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:36:45.934766 (+ 1224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:36:45.934777 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:36:45.935826 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:36:45.935830 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:36:45.935933 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:36:45.935969 (+ 36us) client_negotiation.cc:769] Sending connection context
0809 09:36:45.936110 (+ 141us) client_negotiation.cc:240] Negotiation successful
0809 09:36:45.936224 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":523,"spinlock_wait_cycles":1408,"thread_start_us":86,"threads_started":1}
I20260809 09:36:45.936565 17707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:36:45.931092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51248 (local address 127.20.125.130:40551)
0809 09:36:45.931446 (+ 354us) server_negotiation.cc:206] Beginning negotiation
0809 09:36:45.931461 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 09:36:45.932153 (+ 692us) server_negotiation.cc:407] Connection header received
0809 09:36:45.932241 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:36:45.932247 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:36:45.932421 (+ 174us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:36:45.932575 (+ 154us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 09:36:45.932792 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:36:45.933605 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:36:45.935212 (+ 1607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:36:45.935993 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:36:45.936132 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:36:45.936265 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0809 09:36:45.936328 (+ 63us) server_negotiation.cc:299] Negotiation successful
0809 09:36:45.936423 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":243,"thread_start_us":122,"threads_started":1}
I20260809 09:36:45.936743 17703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:36:45.930498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55170 (local address 127.20.125.129:36585)
0809 09:36:45.931046 (+ 548us) server_negotiation.cc:206] Beginning negotiation
0809 09:36:45.931065 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0809 09:36:45.931091 (+ 26us) server_negotiation.cc:407] Connection header received
0809 09:36:45.931403 (+ 312us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:36:45.931410 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:36:45.931625 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:36:45.931809 (+ 184us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 09:36:45.932316 (+ 507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:36:45.933394 (+ 1078us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:36:45.934903 (+ 1509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:36:45.935696 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:36:45.935827 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:36:45.935962 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0809 09:36:45.936476 (+ 514us) server_negotiation.cc:299] Negotiation successful
0809 09:36:45.936594 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":437,"thread_start_us":84,"threads_started":1}
I20260809 09:36:45.937103 17706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:36:45.930519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.130:40551 (local address 127.0.0.1:51248)
0809 09:36:45.931941 (+ 1422us) negotiation.cc:107] Waiting for socket to connect
0809 09:36:45.931956 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 09:36:45.932050 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:36:45.932565 (+ 515us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:36:45.932568 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:36:45.932589 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 09:36:45.932680 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:36:45.932686 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:36:45.933781 (+ 1095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:36:45.933785 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:36:45.935089 (+ 1304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:36:45.935098 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:36:45.936124 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:36:45.936127 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:36:45.936217 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:36:45.936237 (+ 20us) client_negotiation.cc:769] Sending connection context
0809 09:36:45.936300 (+ 63us) client_negotiation.cc:240] Negotiation successful
0809 09:36:45.936351 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1354,"thread_start_us":94,"threads_started":1}
I20260809 09:36:45.937381 17704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:36:45.930747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.131:38749 (local address 127.0.0.1:46538)
0809 09:36:45.932046 (+ 1299us) negotiation.cc:107] Waiting for socket to connect
0809 09:36:45.932058 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 09:36:45.932135 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:36:45.932730 (+ 595us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:36:45.932735 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:36:45.932757 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 09:36:45.932852 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:36:45.932858 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:36:45.934229 (+ 1371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:36:45.934233 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:36:45.935344 (+ 1111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:36:45.935354 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:36:45.936788 (+ 1434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:36:45.936791 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:36:45.936913 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:36:45.936941 (+ 28us) client_negotiation.cc:769] Sending connection context
0809 09:36:45.937011 (+ 70us) client_negotiation.cc:240] Negotiation successful
0809 09:36:45.937080 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1245,"thread_start_us":90,"threads_started":1}
I20260809 09:36:45.937403 17708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:36:45.931187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46538 (local address 127.20.125.131:38749)
0809 09:36:45.931797 (+ 610us) server_negotiation.cc:206] Beginning negotiation
0809 09:36:45.931809 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 09:36:45.932235 (+ 426us) server_negotiation.cc:407] Connection header received
0809 09:36:45.932306 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:36:45.932310 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:36:45.932453 (+ 143us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:36:45.932583 (+ 130us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 09:36:45.932985 (+ 402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:36:45.934072 (+ 1087us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:36:45.935472 (+ 1400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:36:45.936274 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:36:45.936394 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:36:45.936499 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0809 09:36:45.937180 (+ 681us) server_negotiation.cc:299] Negotiation successful
0809 09:36:45.937273 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":505,"thread_start_us":190,"threads_started":1}
I20260809 09:36:45.939749 17186 tablet_service.cc:1467] Processing CreateTablet for tablet 5508bd12e38c44cf86bac0ba93da5967 (DEFAULT_TABLE table=test-table [id=508bd80329e44fe9a2d9b59c47222896]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 09:36:45.939920 17399 tablet_service.cc:1467] Processing CreateTablet for tablet 5508bd12e38c44cf86bac0ba93da5967 (DEFAULT_TABLE table=test-table [id=508bd80329e44fe9a2d9b59c47222896]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 09:36:45.940317 17587 tablet_service.cc:1467] Processing CreateTablet for tablet 5508bd12e38c44cf86bac0ba93da5967 (DEFAULT_TABLE table=test-table [id=508bd80329e44fe9a2d9b59c47222896]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 09:36:45.940884 17186 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5508bd12e38c44cf86bac0ba93da5967. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 09:36:45.940996 17399 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5508bd12e38c44cf86bac0ba93da5967. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 09:36:45.941250 17587 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5508bd12e38c44cf86bac0ba93da5967. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 09:36:45.947875 17710 tablet_bootstrap.cc:492] T 5508bd12e38c44cf86bac0ba93da5967 P d65bb7049acb48a29e27fc30a4cdc1aa: Bootstrap starting.
I20260809 09:36:45.948897 17711 tablet_bootstrap.cc:492] T 5508bd12e38c44cf86bac0ba93da5967 P e578eee054a646149aa53b8fd83c960a: Bootstrap starting.
I20260809 09:36:45.950393 17710 tablet_bootstrap.cc:654] T 5508bd12e38c44cf86bac0ba93da5967 P d65bb7049acb48a29e27fc30a4cdc1aa: Neither blocks nor log segments found. Creating new log.
I20260809 09:36:45.950631 17711 tablet_bootstrap.cc:654] T 5508bd12e38c44cf86bac0ba93da5967 P e578eee054a646149aa53b8fd83c960a: Neither blocks nor log segments found. Creating new log.
I20260809 09:36:45.950912 17712 tablet_bootstrap.cc:492] T 5508bd12e38c44cf86bac0ba93da5967 P 25f66191e68f4e6fb6c82b73bbb0306d: Bootstrap starting.
I20260809 09:36:45.951319 17710 log.cc:824] T 5508bd12e38c44cf86bac0ba93da5967 P d65bb7049acb48a29e27fc30a4cdc1aa: Log is configured to *not* fsync() on all Append() calls
I20260809 09:36:45.951972 17711 log.cc:824] T 5508bd12e38c44cf86bac0ba93da5967 P e578eee054a646149aa53b8fd83c960a: Log is configured to *not* fsync() on all Append() calls
I20260809 09:36:45.953167 17712 tablet_bootstrap.cc:654] T 5508bd12e38c44cf86bac0ba93da5967 P 25f66191e68f4e6fb6c82b73bbb0306d: Neither blocks nor log segments found. Creating new log.
I20260809 09:36:45.953491 17710 tablet_bootstrap.cc:492] T 5508bd12e38c44cf86bac0ba93da5967 P d65bb7049acb48a29e27fc30a4cdc1aa: No bootstrap required, opened a new log
I20260809 09:36:45.953696 17710 ts_tablet_manager.cc:1397] T 5508bd12e38c44cf86bac0ba93da5967 P d65bb7049acb48a29e27fc30a4cdc1aa: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260809 09:36:45.954077 17711 tablet_bootstrap.cc:492] T 5508bd12e38c44cf86bac0ba93da5967 P e578eee054a646149aa53b8fd83c960a: No bootstrap required, opened a new log
I20260809 09:36:45.954298 17711 ts_tablet_manager.cc:1397] T 5508bd12e38c44cf86bac0ba93da5967 P e578eee054a646149aa53b8fd83c960a: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.005s
I20260809 09:36:45.954550 17712 log.cc:824] T 5508bd12e38c44cf86bac0ba93da5967 P 25f66191e68f4e6fb6c82b73bbb0306d: Log is configured to *not* fsync() on all Append() calls
I20260809 09:36:45.956326 17712 tablet_bootstrap.cc:492] T 5508bd12e38c44cf86bac0ba93da5967 P 25f66191e68f4e6fb6c82b73bbb0306d: No bootstrap required, opened a new log
I20260809 09:36:45.956532 17712 ts_tablet_manager.cc:1397] T 5508bd12e38c44cf86bac0ba93da5967 P 25f66191e68f4e6fb6c82b73bbb0306d: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260809 09:36:45.958353 17710 raft_consensus.cc:348] T 5508bd12e38c44cf86bac0ba93da5967 P d65bb7049acb48a29e27fc30a4cdc1aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d65bb7049acb48a29e27fc30a4cdc1aa" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 36585 } } peers { permanent_uuid: "e578eee054a646149aa53b8fd83c960a" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 40551 } } peers { permanent_uuid: "25f66191e68f4e6fb6c82b73bbb0306d" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 38749 } }
I20260809 09:36:45.958353 17711 raft_consensus.cc:348] T 5508bd12e38c44cf86bac0ba93da5967 P e578eee054a646149aa53b8fd83c960a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d65bb7049acb48a29e27fc30a4cdc1aa" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 36585 } } peers { permanent_uuid: "e578eee054a646149aa53b8fd83c960a" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 40551 } } peers { permanent_uuid: "25f66191e68f4e6fb6c82b73bbb0306d" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 38749 } }
I20260809 09:36:45.958667 17710 raft_consensus.cc:374] T 5508bd12e38c44cf86bac0ba93da5967 P d65bb7049acb48a29e27fc30a4cdc1aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 09:36:45.958662 17711 raft_consensus.cc:374] T 5508bd12e38c44cf86bac0ba93da5967 P e578eee054a646149aa53b8fd83c960a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 09:36:45.958726 17711 raft_consensus.cc:729] T 5508bd12e38c44cf86bac0ba93da5967 P e578eee054a646149aa53b8fd83c960a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e578eee054a646149aa53b8fd83c960a, State: Initialized, Role: FOLLOWER
I20260809 09:36:45.958726 17710 raft_consensus.cc:729] T 5508bd12e38c44cf86bac0ba93da5967 P d65bb7049acb48a29e27fc30a4cdc1aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d65bb7049acb48a29e27fc30a4cdc1aa, State: Initialized, Role: FOLLOWER
I20260809 09:36:45.959199 17710 consensus_queue.cc:260] T 5508bd12e38c44cf86bac0ba93da5967 P d65bb7049acb48a29e27fc30a4cdc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d65bb7049acb48a29e27fc30a4cdc1aa" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 36585 } } peers { permanent_uuid: "e578eee054a646149aa53b8fd83c960a" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 40551 } } peers { permanent_uuid: "25f66191e68f4e6fb6c82b73bbb0306d" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 38749 } }
I20260809 09:36:45.959199 17711 consensus_queue.cc:260] T 5508bd12e38c44cf86bac0ba93da5967 P e578eee054a646149aa53b8fd83c960a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d65bb7049acb48a29e27fc30a4cdc1aa" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 36585 } } peers { permanent_uuid: "e578eee054a646149aa53b8fd83c960a" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 40551 } } peers { permanent_uuid: "25f66191e68f4e6fb6c82b73bbb0306d" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 38749 } }
I20260809 09:36:45.959769 17712 raft_consensus.cc:348] T 5508bd12e38c44cf86bac0ba93da5967 P 25f66191e68f4e6fb6c82b73bbb0306d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d65bb7049acb48a29e27fc30a4cdc1aa" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 36585 } } peers { permanent_uuid: "e578eee054a646149aa53b8fd83c960a" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 40551 } } peers { permanent_uuid: "25f66191e68f4e6fb6c82b73bbb0306d" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 38749 } }
I20260809 09:36:45.960027 17712 raft_consensus.cc:374] T 5508bd12e38c44cf86bac0ba93da5967 P 25f66191e68f4e6fb6c82b73bbb0306d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 09:36:45.960090 17712 raft_consensus.cc:729] T 5508bd12e38c44cf86bac0ba93da5967 P 25f66191e68f4e6fb6c82b73bbb0306d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25f66191e68f4e6fb6c82b73bbb0306d, State: Initialized, Role: FOLLOWER
I20260809 09:36:45.960243 17710 ts_tablet_manager.cc:1428] T 5508bd12e38c44cf86bac0ba93da5967 P d65bb7049acb48a29e27fc30a4cdc1aa: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260809 09:36:45.960243 17711 ts_tablet_manager.cc:1428] T 5508bd12e38c44cf86bac0ba93da5967 P e578eee054a646149aa53b8fd83c960a: Time spent starting tablet: real 0.006s user 0.003s sys 0.002s
I20260809 09:36:45.960608 17491 heartbeater.cc:503] Master 127.20.125.190:45703 was elected leader, sending a full tablet report...
I20260809 09:36:45.960609 17291 heartbeater.cc:503] Master 127.20.125.190:45703 was elected leader, sending a full tablet report...
I20260809 09:36:45.961470 17712 consensus_queue.cc:260] T 5508bd12e38c44cf86bac0ba93da5967 P 25f66191e68f4e6fb6c82b73bbb0306d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d65bb7049acb48a29e27fc30a4cdc1aa" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 36585 } } peers { permanent_uuid: "e578eee054a646149aa53b8fd83c960a" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 40551 } } peers { permanent_uuid: "25f66191e68f4e6fb6c82b73bbb0306d" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 38749 } }
I20260809 09:36:45.962217 17688 heartbeater.cc:503] Master 127.20.125.190:45703 was elected leader, sending a full tablet report...
I20260809 09:36:45.962497 17712 ts_tablet_manager.cc:1428] T 5508bd12e38c44cf86bac0ba93da5967 P 25f66191e68f4e6fb6c82b73bbb0306d: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260809 09:36:46.069011 17716 raft_consensus.cc:482] T 5508bd12e38c44cf86bac0ba93da5967 P e578eee054a646149aa53b8fd83c960a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 09:36:46.069276 17716 raft_consensus.cc:504] T 5508bd12e38c44cf86bac0ba93da5967 P e578eee054a646149aa53b8fd83c960a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d65bb7049acb48a29e27fc30a4cdc1aa" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 36585 } } peers { permanent_uuid: "e578eee054a646149aa53b8fd83c960a" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 40551 } } peers { permanent_uuid: "25f66191e68f4e6fb6c82b73bbb0306d" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 38749 } }
I20260809 09:36:46.070585 17716 leader_election.cc:288] T 5508bd12e38c44cf86bac0ba93da5967 P e578eee054a646149aa53b8fd83c960a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d65bb7049acb48a29e27fc30a4cdc1aa (127.20.125.129:36585), 25f66191e68f4e6fb6c82b73bbb0306d (127.20.125.131:38749)
I20260809 09:36:46.074220 17708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:36:46.070963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.130:53307 (local address 127.20.125.131:38749)
0809 09:36:46.071086 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0809 09:36:46.071091 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 09:36:46.071105 (+ 14us) server_negotiation.cc:407] Connection header received
0809 09:36:46.071266 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:36:46.071271 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:36:46.071324 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:36:46.071413 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 09:36:46.071701 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:36:46.072209 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:36:46.073254 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:36:46.073652 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:36:46.073788 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:36:46.073844 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0809 09:36:46.074028 (+ 184us) server_negotiation.cc:299] Negotiation successful
0809 09:36:46.074100 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260809 09:36:46.074219 17493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:36:46.070835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.131:38749 (local address 127.20.125.130:53307)
0809 09:36:46.071021 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0809 09:36:46.071037 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 09:36:46.071131 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:36:46.071431 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:36:46.071434 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:36:46.071454 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 09:36:46.071579 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:36:46.071584 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:36:46.072326 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:36:46.072331 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:36:46.073118 (+ 787us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:36:46.073128 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:36:46.073780 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:36:46.073783 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:36:46.073890 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:36:46.073918 (+ 28us) client_negotiation.cc:769] Sending connection context
0809 09:36:46.074008 (+ 90us) client_negotiation.cc:240] Negotiation successful
0809 09:36:46.074081 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"mutex_wait_us":1}
I20260809 09:36:46.074748 17703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:36:46.070913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.130:59983 (local address 127.20.125.129:36585)
0809 09:36:46.071049 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0809 09:36:46.071054 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 09:36:46.071356 (+ 302us) server_negotiation.cc:407] Connection header received
0809 09:36:46.071690 (+ 334us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:36:46.071694 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:36:46.071750 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:36:46.071859 (+ 109us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 09:36:46.072169 (+ 310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:36:46.072664 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:36:46.073716 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:36:46.074121 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:36:46.074253 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:36:46.074301 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0809 09:36:46.074540 (+ 239us) server_negotiation.cc:299] Negotiation successful
0809 09:36:46.074626 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260809 09:36:46.074856 17720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:36:46.070838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.129:36585 (local address 127.20.125.130:59983)
0809 09:36:46.071245 (+ 407us) negotiation.cc:107] Waiting for socket to connect
0809 09:36:46.071257 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 09:36:46.071395 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:36:46.071873 (+ 478us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:36:46.071876 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:36:46.071893 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 09:36:46.072038 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:36:46.072043 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:36:46.072807 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:36:46.072813 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:36:46.073584 (+ 771us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:36:46.073593 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:36:46.074247 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:36:46.074251 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:36:46.074364 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:36:46.074391 (+ 27us) client_negotiation.cc:769] Sending connection context
0809 09:36:46.074625 (+ 234us) client_negotiation.cc:240] Negotiation successful
0809 09:36:46.074696 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":347,"mutex_wait_us":15,"thread_start_us":127,"threads_started":1}
I20260809 09:36:46.074956 17617 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5508bd12e38c44cf86bac0ba93da5967" candidate_uuid: "e578eee054a646149aa53b8fd83c960a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25f66191e68f4e6fb6c82b73bbb0306d" is_pre_election: true
I20260809 09:36:46.075300 17617 raft_consensus.cc:2454] T 5508bd12e38c44cf86bac0ba93da5967 P 25f66191e68f4e6fb6c82b73bbb0306d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e578eee054a646149aa53b8fd83c960a in term 0.
I20260809 09:36:46.075435 17216 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5508bd12e38c44cf86bac0ba93da5967" candidate_uuid: "e578eee054a646149aa53b8fd83c960a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d65bb7049acb48a29e27fc30a4cdc1aa" is_pre_election: true
I20260809 09:36:46.075729 17216 raft_consensus.cc:2454] T 5508bd12e38c44cf86bac0ba93da5967 P d65bb7049acb48a29e27fc30a4cdc1aa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e578eee054a646149aa53b8fd83c960a in term 0.
I20260809 09:36:46.075824 17320 leader_election.cc:302] T 5508bd12e38c44cf86bac0ba93da5967 P e578eee054a646149aa53b8fd83c960a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 25f66191e68f4e6fb6c82b73bbb0306d, e578eee054a646149aa53b8fd83c960a; no voters:
I20260809 09:36:46.076189 17716 raft_consensus.cc:2781] T 5508bd12e38c44cf86bac0ba93da5967 P e578eee054a646149aa53b8fd83c960a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 09:36:46.076288 17716 raft_consensus.cc:482] T 5508bd12e38c44cf86bac0ba93da5967 P e578eee054a646149aa53b8fd83c960a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 09:36:46.076335 17716 raft_consensus.cc:3037] T 5508bd12e38c44cf86bac0ba93da5967 P e578eee054a646149aa53b8fd83c960a [term 0 FOLLOWER]: Advancing to term 1
I20260809 09:36:46.077497 17716 raft_consensus.cc:504] T 5508bd12e38c44cf86bac0ba93da5967 P e578eee054a646149aa53b8fd83c960a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d65bb7049acb48a29e27fc30a4cdc1aa" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 36585 } } peers { permanent_uuid: "e578eee054a646149aa53b8fd83c960a" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 40551 } } peers { permanent_uuid: "25f66191e68f4e6fb6c82b73bbb0306d" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 38749 } }
I20260809 09:36:46.077857 17716 leader_election.cc:288] T 5508bd12e38c44cf86bac0ba93da5967 P e578eee054a646149aa53b8fd83c960a [CANDIDATE]: Term 1 election: Requested vote from peers d65bb7049acb48a29e27fc30a4cdc1aa (127.20.125.129:36585), 25f66191e68f4e6fb6c82b73bbb0306d (127.20.125.131:38749)
I20260809 09:36:46.078162 17216 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5508bd12e38c44cf86bac0ba93da5967" candidate_uuid: "e578eee054a646149aa53b8fd83c960a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d65bb7049acb48a29e27fc30a4cdc1aa"
I20260809 09:36:46.078289 17216 raft_consensus.cc:3037] T 5508bd12e38c44cf86bac0ba93da5967 P d65bb7049acb48a29e27fc30a4cdc1aa [term 0 FOLLOWER]: Advancing to term 1
I20260809 09:36:46.078306 17617 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5508bd12e38c44cf86bac0ba93da5967" candidate_uuid: "e578eee054a646149aa53b8fd83c960a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25f66191e68f4e6fb6c82b73bbb0306d"
I20260809 09:36:46.078454 17617 raft_consensus.cc:3037] T 5508bd12e38c44cf86bac0ba93da5967 P 25f66191e68f4e6fb6c82b73bbb0306d [term 0 FOLLOWER]: Advancing to term 1
I20260809 09:36:46.079452 17216 raft_consensus.cc:2454] T 5508bd12e38c44cf86bac0ba93da5967 P d65bb7049acb48a29e27fc30a4cdc1aa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e578eee054a646149aa53b8fd83c960a in term 1.
I20260809 09:36:46.079618 17617 raft_consensus.cc:2454] T 5508bd12e38c44cf86bac0ba93da5967 P 25f66191e68f4e6fb6c82b73bbb0306d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e578eee054a646149aa53b8fd83c960a in term 1.
I20260809 09:36:46.079802 17321 leader_election.cc:302] T 5508bd12e38c44cf86bac0ba93da5967 P e578eee054a646149aa53b8fd83c960a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d65bb7049acb48a29e27fc30a4cdc1aa, e578eee054a646149aa53b8fd83c960a; no voters:
I20260809 09:36:46.079979 17716 raft_consensus.cc:2781] T 5508bd12e38c44cf86bac0ba93da5967 P e578eee054a646149aa53b8fd83c960a [term 1 FOLLOWER]: Leader election won for term 1
I20260809 09:36:46.080227 17716 raft_consensus.cc:686] T 5508bd12e38c44cf86bac0ba93da5967 P e578eee054a646149aa53b8fd83c960a [term 1 LEADER]: Becoming Leader. State: Replica: e578eee054a646149aa53b8fd83c960a, State: Running, Role: LEADER
I20260809 09:36:46.080557 17716 consensus_queue.cc:237] T 5508bd12e38c44cf86bac0ba93da5967 P e578eee054a646149aa53b8fd83c960a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d65bb7049acb48a29e27fc30a4cdc1aa" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 36585 } } peers { permanent_uuid: "e578eee054a646149aa53b8fd83c960a" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 40551 } } peers { permanent_uuid: "25f66191e68f4e6fb6c82b73bbb0306d" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 38749 } }
I20260809 09:36:46.083966 17000 catalog_manager.cc:5495] T 5508bd12e38c44cf86bac0ba93da5967 P e578eee054a646149aa53b8fd83c960a reported cstate change: term changed from 0 to 1, leader changed from <none> to e578eee054a646149aa53b8fd83c960a (127.20.125.130). New cstate: current_term: 1 leader_uuid: "e578eee054a646149aa53b8fd83c960a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d65bb7049acb48a29e27fc30a4cdc1aa" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 36585 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e578eee054a646149aa53b8fd83c960a" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 40551 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "25f66191e68f4e6fb6c82b73bbb0306d" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 38749 } health_report { overall_health: UNKNOWN } } }
W20260809 09:36:46.126405 17689 tablet.cc:2369] T 5508bd12e38c44cf86bac0ba93da5967 P 25f66191e68f4e6fb6c82b73bbb0306d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 09:36:46.128319 17707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:36:46.124148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51256 (local address 127.20.125.130:40551)
0809 09:36:46.124395 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0809 09:36:46.124401 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 09:36:46.124416 (+ 15us) server_negotiation.cc:407] Connection header received
0809 09:36:46.124603 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:36:46.124605 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:36:46.124667 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:36:46.124791 (+ 124us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 09:36:46.125071 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:36:46.125585 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:36:46.126664 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:36:46.126909 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:36:46.127046 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:36:46.127391 (+ 345us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 09:36:46.127806 (+ 415us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 09:36:46.127912 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0809 09:36:46.128010 (+ 98us) server_negotiation.cc:299] Negotiation successful
0809 09:36:46.128096 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153}
I20260809 09:36:46.134506 17617 raft_consensus.cc:1261] T 5508bd12e38c44cf86bac0ba93da5967 P 25f66191e68f4e6fb6c82b73bbb0306d [term 1 FOLLOWER]: Refusing update from remote peer e578eee054a646149aa53b8fd83c960a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 09:36:46.134559 17216 raft_consensus.cc:1261] T 5508bd12e38c44cf86bac0ba93da5967 P d65bb7049acb48a29e27fc30a4cdc1aa [term 1 FOLLOWER]: Refusing update from remote peer e578eee054a646149aa53b8fd83c960a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 09:36:46.135277 17716 consensus_queue.cc:1035] T 5508bd12e38c44cf86bac0ba93da5967 P e578eee054a646149aa53b8fd83c960a [LEADER]: Connected to new peer: Peer: permanent_uuid: "25f66191e68f4e6fb6c82b73bbb0306d" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 38749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 09:36:46.135478 17721 consensus_queue.cc:1035] T 5508bd12e38c44cf86bac0ba93da5967 P e578eee054a646149aa53b8fd83c960a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d65bb7049acb48a29e27fc30a4cdc1aa" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 36585 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 09:36:46.142508 17726 mvcc.cc:205] Tried to move back new op lower bound from 7316554572319035392 to 7316554572108484608. Current Snapshot: MvccSnapshot[applied={T|T < 7316554572319035392}]
I20260809 09:36:46.153306 17001 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48044:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 09:36:46.153483 17001 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48044:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 09:36:46.156538 17001 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 13463be3de9b4ec9bd6d7ecf40625d98: Sending DeleteTablet for 3 replicas of tablet 5508bd12e38c44cf86bac0ba93da5967
I20260809 09:36:46.157455 17186 tablet_service.cc:1514] Processing DeleteTablet for tablet 5508bd12e38c44cf86bac0ba93da5967 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 09:36:46 UTC) from {username='kudu', principal='kudu/127.20.125.190@KRBTEST.COM'} at 127.0.0.1:55170
I20260809 09:36:46.157558 17399 tablet_service.cc:1514] Processing DeleteTablet for tablet 5508bd12e38c44cf86bac0ba93da5967 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 09:36:46 UTC) from {username='kudu', principal='kudu/127.20.125.190@KRBTEST.COM'} at 127.0.0.1:51248
I20260809 09:36:46.157697 17587 tablet_service.cc:1514] Processing DeleteTablet for tablet 5508bd12e38c44cf86bac0ba93da5967 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 09:36:46 UTC) from {username='kudu', principal='kudu/127.20.125.190@KRBTEST.COM'} at 127.0.0.1:46538
I20260809 09:36:46.158068 17742 tablet_replica.cc:330] T 5508bd12e38c44cf86bac0ba93da5967 P e578eee054a646149aa53b8fd83c960a: stopping tablet replica
I20260809 09:36:46.158123 17586 tablet_service.cc:1514] Processing DeleteTablet for tablet 5508bd12e38c44cf86bac0ba93da5967 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 09:36:46 UTC) from {username='kudu', principal='kudu/127.20.125.190@KRBTEST.COM'} at 127.0.0.1:46538
I20260809 09:36:46.158389 17742 raft_consensus.cc:2229] T 5508bd12e38c44cf86bac0ba93da5967 P e578eee054a646149aa53b8fd83c960a [term 1 LEADER]: Raft consensus shutting down.
I20260809 09:36:46.158625 20982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu with pid 17094
I20260809 09:36:46.158766 17742 raft_consensus.cc:2258] T 5508bd12e38c44cf86bac0ba93da5967 P e578eee054a646149aa53b8fd83c960a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 09:36:46.161166 17744 tablet_replica.cc:330] T 5508bd12e38c44cf86bac0ba93da5967 P 25f66191e68f4e6fb6c82b73bbb0306d: stopping tablet replica
I20260809 09:36:46.161517 17744 raft_consensus.cc:2229] T 5508bd12e38c44cf86bac0ba93da5967 P 25f66191e68f4e6fb6c82b73bbb0306d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 09:36:46.161871 17744 raft_consensus.cc:2258] T 5508bd12e38c44cf86bac0ba93da5967 P 25f66191e68f4e6fb6c82b73bbb0306d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 09:36:46.163997 17744 ts_tablet_manager.cc:1905] T 5508bd12e38c44cf86bac0ba93da5967 P 25f66191e68f4e6fb6c82b73bbb0306d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 09:36:46.167842 17744 ts_tablet_manager.cc:1918] T 5508bd12e38c44cf86bac0ba93da5967 P 25f66191e68f4e6fb6c82b73bbb0306d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 09:36:46.168079 17744 log.cc:1196] T 5508bd12e38c44cf86bac0ba93da5967 P 25f66191e68f4e6fb6c82b73bbb0306d: Deleting WAL directory at /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786267941944493-20982-0/minicluster-data/ts-2/wal/wals/5508bd12e38c44cf86bac0ba93da5967
I20260809 09:36:46.168675 17744 ts_tablet_manager.cc:1939] T 5508bd12e38c44cf86bac0ba93da5967 P 25f66191e68f4e6fb6c82b73bbb0306d: Deleting consensus metadata
I20260809 09:36:46.170679 16982 catalog_manager.cc:4841] TS 25f66191e68f4e6fb6c82b73bbb0306d (127.20.125.131:38749): tablet 5508bd12e38c44cf86bac0ba93da5967 (table test-table [id=508bd80329e44fe9a2d9b59c47222896]) successfully deleted
W20260809 09:36:46.171075 16982 catalog_manager.cc:4808] TS 25f66191e68f4e6fb6c82b73bbb0306d (127.20.125.131:38749): delete failed for tablet 5508bd12e38c44cf86bac0ba93da5967 because the tablet was not found. No further retry: Not found: Tablet not found: 5508bd12e38c44cf86bac0ba93da5967
I20260809 09:36:46.172137 17742 ts_tablet_manager.cc:1905] T 5508bd12e38c44cf86bac0ba93da5967 P e578eee054a646149aa53b8fd83c960a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 09:36:46.174435 17742 ts_tablet_manager.cc:1918] T 5508bd12e38c44cf86bac0ba93da5967 P e578eee054a646149aa53b8fd83c960a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 09:36:46.174621 17742 log.cc:1196] T 5508bd12e38c44cf86bac0ba93da5967 P e578eee054a646149aa53b8fd83c960a: Deleting WAL directory at /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786267941944493-20982-0/minicluster-data/ts-1/wal/wals/5508bd12e38c44cf86bac0ba93da5967
I20260809 09:36:46.174947 17742 ts_tablet_manager.cc:1939] T 5508bd12e38c44cf86bac0ba93da5967 P e578eee054a646149aa53b8fd83c960a: Deleting consensus metadata
I20260809 09:36:46.176035 16981 catalog_manager.cc:4841] TS e578eee054a646149aa53b8fd83c960a (127.20.125.130:40551): tablet 5508bd12e38c44cf86bac0ba93da5967 (table test-table [id=508bd80329e44fe9a2d9b59c47222896]) successfully deleted
W20260809 09:36:46.178556 16983 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.20.125.129:36585 (error 108)
I20260809 09:36:46.178752 20982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu with pid 17298
I20260809 09:36:46.183408 17706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:36:46.182998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.129:36585 (local address 127.0.0.1:55176)
0809 09:36:46.183223 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0809 09:36:46.183298 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.129:36585: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131}
W20260809 09:36:46.183604 16983 catalog_manager.cc:4584] TS d65bb7049acb48a29e27fc30a4cdc1aa (127.20.125.129:36585): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 5508bd12e38c44cf86bac0ba93da5967: Network error: Client connection negotiation failed: client connection to 127.20.125.129:36585: connect: Connection refused (error 111)
I20260809 09:36:46.189384 20982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu with pid 17496
I20260809 09:36:46.199736 20982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu with pid 16956
2026-08-09T09:36:46Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2729 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-09T09:36:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T09:36:46Z Disabled control of system clock
I20260809 09:36:46.240185 20982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
/tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786267941944493-20982-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786267941944493-20982-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786267941944493-20982-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786267941944493-20982-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.125.190:41497
--webserver_interface=127.20.125.190
--webserver_port=0
--builtin_ntp_servers=127.20.125.148:43133
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.125.190:41497
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 09:36:46.357992 17755 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 09:36:46.358251 17755 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 09:36:46.358285 17755 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 09:36:46.362231 17755 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 09:36:46.362290 17755 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 09:36:46.362306 17755 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 09:36:46.362322 17755 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 09:36:46.362337 17755 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 09:36:46.367246 17755 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.125.148:43133
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786267941944493-20982-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786267941944493-20982-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.125.190:41497
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.125.190:41497
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786267941944493-20982-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.125.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17755
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786267941944493-20982-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 08:38:55 UTC on ccycloud.quasar-njjleu.root.comops.site
build id 4575935
I20260809 09:36:46.368299 17755 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 09:36:46.369638 17755 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 09:36:46.378110 17765 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:36:46.378124 17769 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:36:46.378304 17755 server_base.cc:1034] running on GCE node
W20260809 09:36:46.378124 17766 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:36:46.379009 17755 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 09:36:46.380100 17755 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 09:36:46.381285 17755 hybrid_clock.cc:648] HybridClock initialized: now 1786268206381266 us; error 30 us; skew 500 ppm
I20260809 09:36:46.389170 17755 webserver.cc:460] Webserver started at http://127.20.125.190:44701/ using document root <none> and password file <none>
I20260809 09:36:46.389775 17755 fs_manager.cc:359] Metadata directory not provided
I20260809 09:36:46.389843 17755 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 09:36:46.390072 17755 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 09:36:46.392189 17755 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786267941944493-20982-0/minicluster-data/master-0/data/instance:
uuid: "8958145386e244499f9adca3756f17c8"
format_stamp: "Formatted at 2026-08-09 09:36:46 on dist-test-slave-7kzw"
server_key: "bf61ffea1ad84c7bf4f5ff42532c28d5"
server_key_iv: "0813d94033c8adc25eaa4cdf0712474d"
server_key_version: "encryptionkey@0"
I20260809 09:36:46.392776 17755 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786267941944493-20982-0/minicluster-data/master-0/wal/instance:
uuid: "8958145386e244499f9adca3756f17c8"
format_stamp: "Formatted at 2026-08-09 09:36:46 on dist-test-slave-7kzw"
server_key: "bf61ffea1ad84c7bf4f5ff42532c28d5"
server_key_iv: "0813d94033c8adc25eaa4cdf0712474d"
server_key_version: "encryptionkey@0"
I20260809 09:36:46.396536 17755 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.002s
I20260809 09:36:46.399052 17779 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 09:36:46.400277 17755 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 09:36:46.400426 17755 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786267941944493-20982-0/minicluster-data/master-0/data,/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786267941944493-20982-0/minicluster-data/master-0/wal
uuid: "8958145386e244499f9adca3756f17c8"
format_stamp: "Formatted at 2026-08-09 09:36:46 on dist-test-slave-7kzw"
server_key: "bf61ffea1ad84c7bf4f5ff42532c28d5"
server_key_iv: "0813d94033c8adc25eaa4cdf0712474d"
server_key_version: "encryptionkey@0"
I20260809 09:36:46.400537 17755 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786267941944493-20982-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786267941944493-20982-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786267941944493-20982-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 09:36:46.419890 17755 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 09:36:46.420578 17755 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 09:36:46.420794 17755 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 09:36:46.428416 17755 rpc_server.cc:307] RPC server started. Bound to: 127.20.125.190:41497
I20260809 09:36:46.428457 17856 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.125.190:41497 every 8 connection(s)
I20260809 09:36:46.429364 17755 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786267941944493-20982-0/minicluster-data/master-0/data/info.pb
I20260809 09:36:46.432574 17858 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 09:36:46.437067 20982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu as pid 17755
I20260809 09:36:46.437228 20982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786267941944493-20982-0/minicluster-data/master-0/wal/instance
I20260809 09:36:46.437523 20982 external_mini_cluster.cc:1428] Setting key 954bd5c030f26651dedfd568790602ff
I20260809 09:36:46.438390 17858 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8958145386e244499f9adca3756f17c8: Bootstrap starting.
I20260809 09:36:46.440938 17858 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8958145386e244499f9adca3756f17c8: Neither blocks nor log segments found. Creating new log.
I20260809 09:36:46.441728 17858 log.cc:824] T 00000000000000000000000000000000 P 8958145386e244499f9adca3756f17c8: Log is configured to *not* fsync() on all Append() calls
I20260809 09:36:46.443642 17858 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8958145386e244499f9adca3756f17c8: No bootstrap required, opened a new log
I20260809 09:36:46.446800 17862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:36:46.439014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51574 (local address 127.20.125.190:41497)
0809 09:36:46.439707 (+ 693us) server_negotiation.cc:206] Beginning negotiation
0809 09:36:46.439723 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 09:36:46.439759 (+ 36us) server_negotiation.cc:407] Connection header received
0809 09:36:46.440389 (+ 630us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:36:46.440406 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:36:46.440788 (+ 382us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:36:46.441135 (+ 347us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:36:46.441582 (+ 447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:36:46.442627 (+ 1045us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:36:46.443495 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:36:46.444014 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:36:46.444195 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:36:46.445074 (+ 879us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:36:46.445080 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:36:46.445092 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 09:36:46.445468 (+ 376us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:36:46.445522 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 09:36:46.445541 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:36:46.445650 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0809 09:36:46.445788 (+ 138us) server_negotiation.cc:299] Negotiation successful
0809 09:36:46.446032 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":314,"thread_start_us":136,"threads_started":1}
I20260809 09:36:46.447355 17858 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8958145386e244499f9adca3756f17c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8958145386e244499f9adca3756f17c8" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 41497 } }
I20260809 09:36:46.447607 17858 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8958145386e244499f9adca3756f17c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 09:36:46.447674 17858 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8958145386e244499f9adca3756f17c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8958145386e244499f9adca3756f17c8, State: Initialized, Role: FOLLOWER
I20260809 09:36:46.448179 17858 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8958145386e244499f9adca3756f17c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8958145386e244499f9adca3756f17c8" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 41497 } }
I20260809 09:36:46.448342 17858 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8958145386e244499f9adca3756f17c8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 09:36:46.448415 17858 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8958145386e244499f9adca3756f17c8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 09:36:46.448559 17858 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8958145386e244499f9adca3756f17c8 [term 0 FOLLOWER]: Advancing to term 1
I20260809 09:36:46.449584 17858 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8958145386e244499f9adca3756f17c8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8958145386e244499f9adca3756f17c8" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 41497 } }
I20260809 09:36:46.449932 17858 leader_election.cc:302] T 00000000000000000000000000000000 P 8958145386e244499f9adca3756f17c8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8958145386e244499f9adca3756f17c8; no voters:
I20260809 09:36:46.450273 17858 leader_election.cc:288] T 00000000000000000000000000000000 P 8958145386e244499f9adca3756f17c8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 09:36:46.450350 17864 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8958145386e244499f9adca3756f17c8 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 09:36:46.450647 17864 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8958145386e244499f9adca3756f17c8 [term 1 LEADER]: Becoming Leader. State: Replica: 8958145386e244499f9adca3756f17c8, State: Running, Role: LEADER
I20260809 09:36:46.450959 17864 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8958145386e244499f9adca3756f17c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8958145386e244499f9adca3756f17c8" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 41497 } }
I20260809 09:36:46.451524 17858 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8958145386e244499f9adca3756f17c8 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 09:36:46.452559 17867 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8958145386e244499f9adca3756f17c8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8958145386e244499f9adca3756f17c8. Latest consensus state: current_term: 1 leader_uuid: "8958145386e244499f9adca3756f17c8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8958145386e244499f9adca3756f17c8" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 41497 } } }
I20260809 09:36:46.452733 17866 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8958145386e244499f9adca3756f17c8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8958145386e244499f9adca3756f17c8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8958145386e244499f9adca3756f17c8" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 41497 } } }
I20260809 09:36:46.452852 17866 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8958145386e244499f9adca3756f17c8 [sys.catalog]: This master's current role is: LEADER
I20260809 09:36:46.452781 17867 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8958145386e244499f9adca3756f17c8 [sys.catalog]: This master's current role is: LEADER
I20260809 09:36:46.453258 17877 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 09:36:46.456215 17877 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 09:36:46.461733 17877 catalog_manager.cc:1347] Generated new cluster ID: 7b0325438b574902b24eb75ff11ad7be
I20260809 09:36:46.461804 17877 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 09:36:46.468827 17877 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 09:36:46.469722 17877 catalog_manager.cc:1504] Loading token signing keys...
I20260809 09:36:46.477650 17877 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8958145386e244499f9adca3756f17c8: Generated new TSK 0
I20260809 09:36:46.478434 17877 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 09:36:46.486864 20982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
/tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786267941944493-20982-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786267941944493-20982-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786267941944493-20982-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786267941944493-20982-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.125.129:0
--local_ip_for_outbound_sockets=127.20.125.129
--webserver_interface=127.20.125.129
--webserver_port=0
--tserver_master_addrs=127.20.125.190:41497
--builtin_ntp_servers=127.20.125.148:43133
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 09:36:46.607581 17890 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 09:36:46.607882 17890 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 09:36:46.607955 17890 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 09:36:46.611958 17890 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 09:36:46.612038 17890 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 09:36:46.612128 17890 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.125.129
I20260809 09:36:46.616796 17890 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.125.148:43133
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786267941944493-20982-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786267941944493-20982-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.125.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786267941944493-20982-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.125.129
--webserver_port=0
--tserver_master_addrs=127.20.125.190:41497
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17890
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.125.129
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786267941944493-20982-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 08:38:55 UTC on ccycloud.quasar-njjleu.root.comops.site
build id 4575935
I20260809 09:36:46.617869 17890 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 09:36:46.619315 17890 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 09:36:46.628911 17897 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:36:46.628904 17899 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:36:46.628904 17895 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:36:46.629382 17890 server_base.cc:1034] running on GCE node
I20260809 09:36:46.629796 17890 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 09:36:46.630422 17890 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 09:36:46.631641 17890 hybrid_clock.cc:648] HybridClock initialized: now 1786268206631622 us; error 29 us; skew 500 ppm
I20260809 09:36:46.633472 17890 webserver.cc:460] Webserver started at http://127.20.125.129:32775/ using document root <none> and password file <none>
I20260809 09:36:46.634059 17890 fs_manager.cc:359] Metadata directory not provided
I20260809 09:36:46.634155 17890 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 09:36:46.634383 17890 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 09:36:46.636523 17890 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786267941944493-20982-0/minicluster-data/ts-0/data/instance:
uuid: "586b0ffe5d5d42f7a5e36c74a3beb965"
format_stamp: "Formatted at 2026-08-09 09:36:46 on dist-test-slave-7kzw"
server_key: "227643cdb812c72d0c6cda7d6f539159"
server_key_iv: "96f5a93df67e63f8e65e5e64adcc9dd1"
server_key_version: "encryptionkey@0"
I20260809 09:36:46.637104 17890 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786267941944493-20982-0/minicluster-data/ts-0/wal/instance:
uuid: "586b0ffe5d5d42f7a5e36c74a3beb965"
format_stamp: "Formatted at 2026-08-09 09:36:46 on dist-test-slave-7kzw"
server_key: "227643cdb812c72d0c6cda7d6f539159"
server_key_iv: "96f5a93df67e63f8e65e5e64adcc9dd1"
server_key_version: "encryptionkey@0"
I20260809 09:36:46.640795 17890 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.005s
I20260809 09:36:46.643270 17909 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 09:36:46.644461 17890 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.003s
I20260809 09:36:46.644614 17890 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786267941944493-20982-0/minicluster-data/ts-0/data,/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786267941944493-20982-0/minicluster-data/ts-0/wal
uuid: "586b0ffe5d5d42f7a5e36c74a3beb965"
format_stamp: "Formatted at 2026-08-09 09:36:46 on dist-test-slave-7kzw"
server_key: "227643cdb812c72d0c6cda7d6f539159"
server_key_iv: "96f5a93df67e63f8e65e5e64adcc9dd1"
server_key_version: "encryptionkey@0"
I20260809 09:36:46.644731 17890 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786267941944493-20982-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786267941944493-20982-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786267941944493-20982-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 09:36:46.663275 17890 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 09:36:46.663915 17890 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 09:36:46.664116 17890 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 09:36:46.664697 17890 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 09:36:46.665642 17890 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 09:36:46.665715 17890 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 09:36:46.665788 17890 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 09:36:46.665840 17890 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 09:36:46.674913 17890 rpc_server.cc:307] RPC server started. Bound to: 127.20.125.129:33511
I20260809 09:36:46.674937 18080 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.125.129:33511 every 8 connection(s)
I20260809 09:36:46.675799 17890 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786267941944493-20982-0/minicluster-data/ts-0/data/info.pb
I20260809 09:36:46.684481 17862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:36:46.677629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.129:60791 (local address 127.20.125.190:41497)
0809 09:36:46.677779 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0809 09:36:46.677783 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 09:36:46.678832 (+ 1049us) server_negotiation.cc:407] Connection header received
0809 09:36:46.680181 (+ 1349us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:36:46.680184 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:36:46.680230 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:36:46.680306 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:36:46.681123 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:36:46.681670 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:36:46.682755 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:36:46.682978 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:36:46.683056 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:36:46.684076 (+ 1020us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:36:46.684079 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:36:46.684081 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 09:36:46.684093 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:36:46.684113 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 09:36:46.684121 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:36:46.684190 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0809 09:36:46.684315 (+ 125us) server_negotiation.cc:299] Negotiation successful
0809 09:36:46.684365 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260809 09:36:46.684911 20982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu as pid 17890
I20260809 09:36:46.685058 20982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786267941944493-20982-0/minicluster-data/ts-0/wal/instance
I20260809 09:36:46.685348 20982 external_mini_cluster.cc:1428] Setting key 085c69e79238ed072646f0574579bb73
I20260809 09:36:46.685386 18085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:36:46.678021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:41497 (local address 127.20.125.129:60791)
0809 09:36:46.678708 (+ 687us) negotiation.cc:107] Waiting for socket to connect
0809 09:36:46.678749 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 09:36:46.679982 (+ 1233us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:36:46.680467 (+ 485us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:36:46.680473 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:36:46.680551 (+ 78us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:36:46.681006 (+ 455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:36:46.681018 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:36:46.681786 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:36:46.681789 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:36:46.682652 (+ 863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:36:46.682661 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:36:46.683106 (+ 445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:36:46.683109 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:36:46.683251 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:36:46.683844 (+ 593us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 09:36:46.683846 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:36:46.683883 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 09:36:46.683939 (+ 56us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 09:36:46.684198 (+ 259us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 09:36:46.684205 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 09:36:46.684209 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 09:36:46.684297 (+ 88us) client_negotiation.cc:240] Negotiation successful
0809 09:36:46.684492 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":263,"thread_start_us":118,"threads_started":1}
I20260809 09:36:46.686664 18081 heartbeater.cc:347] Connected to a master server at 127.20.125.190:41497
I20260809 09:36:46.686920 18081 heartbeater.cc:464] Registering TS with master...
I20260809 09:36:46.687448 18081 heartbeater.cc:511] Master 127.20.125.190:41497 requested a full tablet report, sending...
I20260809 09:36:46.687708 20982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
/tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786267941944493-20982-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786267941944493-20982-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786267941944493-20982-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786267941944493-20982-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.125.130:0
--local_ip_for_outbound_sockets=127.20.125.130
--webserver_interface=127.20.125.130
--webserver_port=0
--tserver_master_addrs=127.20.125.190:41497
--builtin_ntp_servers=127.20.125.148:43133
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260809 09:36:46.689033 17803 ts_manager.cc:194] Registered new tserver with Master: 586b0ffe5d5d42f7a5e36c74a3beb965 (127.20.125.129:33511)
I20260809 09:36:46.690284 17803 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.20.125.129:60791
W20260809 09:36:46.800390 18087 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 09:36:46.800717 18087 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 09:36:46.800802 18087 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 09:36:46.804883 18087 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 09:36:46.804961 18087 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 09:36:46.805049 18087 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.125.130
I20260809 09:36:46.809762 18087 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.125.148:43133
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786267941944493-20982-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786267941944493-20982-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.125.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786267941944493-20982-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.125.130
--webserver_port=0
--tserver_master_addrs=127.20.125.190:41497
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18087
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.125.130
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786267941944493-20982-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 08:38:55 UTC on ccycloud.quasar-njjleu.root.comops.site
build id 4575935
I20260809 09:36:46.810859 18087 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 09:36:46.812242 18087 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 09:36:46.821998 18094 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:36:46.822116 18095 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:36:46.822046 18097 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:36:46.822365 18087 server_base.cc:1034] running on GCE node
I20260809 09:36:46.822801 18087 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 09:36:46.823441 18087 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 09:36:46.824627 18087 hybrid_clock.cc:648] HybridClock initialized: now 1786268206824611 us; error 29 us; skew 500 ppm
I20260809 09:36:46.826452 18087 webserver.cc:460] Webserver started at http://127.20.125.130:44201/ using document root <none> and password file <none>
I20260809 09:36:46.827107 18087 fs_manager.cc:359] Metadata directory not provided
I20260809 09:36:46.827206 18087 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 09:36:46.827468 18087 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 09:36:46.829540 18087 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786267941944493-20982-0/minicluster-data/ts-1/data/instance:
uuid: "04beac347d7245508c817aa7ff612683"
format_stamp: "Formatted at 2026-08-09 09:36:46 on dist-test-slave-7kzw"
server_key: "21e435350827fef595edcda58c91932d"
server_key_iv: "dd189709d282fd5c97fa1f21b1d19d4d"
server_key_version: "encryptionkey@0"
I20260809 09:36:46.830137 18087 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786267941944493-20982-0/minicluster-data/ts-1/wal/instance:
uuid: "04beac347d7245508c817aa7ff612683"
format_stamp: "Formatted at 2026-08-09 09:36:46 on dist-test-slave-7kzw"
server_key: "21e435350827fef595edcda58c91932d"
server_key_iv: "dd189709d282fd5c97fa1f21b1d19d4d"
server_key_version: "encryptionkey@0"
I20260809 09:36:46.833948 18087 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.003s
I20260809 09:36:46.836534 18105 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 09:36:46.837756 18087 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 09:36:46.837905 18087 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786267941944493-20982-0/minicluster-data/ts-1/data,/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786267941944493-20982-0/minicluster-data/ts-1/wal
uuid: "04beac347d7245508c817aa7ff612683"
format_stamp: "Formatted at 2026-08-09 09:36:46 on dist-test-slave-7kzw"
server_key: "21e435350827fef595edcda58c91932d"
server_key_iv: "dd189709d282fd5c97fa1f21b1d19d4d"
server_key_version: "encryptionkey@0"
I20260809 09:36:46.838023 18087 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786267941944493-20982-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786267941944493-20982-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786267941944493-20982-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 09:36:46.853984 18087 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 09:36:46.854741 18087 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 09:36:46.854959 18087 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 09:36:46.855582 18087 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 09:36:46.856588 18087 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 09:36:46.856662 18087 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 09:36:46.856738 18087 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 09:36:46.856783 18087 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 09:36:46.866169 18087 rpc_server.cc:307] RPC server started. Bound to: 127.20.125.130:46251
I20260809 09:36:46.866197 18277 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.125.130:46251 every 8 connection(s)
I20260809 09:36:46.867118 18087 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786267941944493-20982-0/minicluster-data/ts-1/data/info.pb
I20260809 09:36:46.875964 20982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu as pid 18087
I20260809 09:36:46.876124 20982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786267941944493-20982-0/minicluster-data/ts-1/wal/instance
I20260809 09:36:46.876525 20982 external_mini_cluster.cc:1428] Setting key 0bce1f1f220dd4dfbfc7e78fa6bbb907
I20260809 09:36:46.876803 17862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:36:46.869087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.130:60369 (local address 127.20.125.190:41497)
0809 09:36:46.869260 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0809 09:36:46.869263 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 09:36:46.870333 (+ 1070us) server_negotiation.cc:407] Connection header received
0809 09:36:46.871815 (+ 1482us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:36:46.871818 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:36:46.871880 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:36:46.871992 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:36:46.872879 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:36:46.873417 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:36:46.874597 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:36:46.874824 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:36:46.874942 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:36:46.876168 (+ 1226us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:36:46.876185 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:36:46.876190 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 09:36:46.876207 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:36:46.876230 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 09:36:46.876241 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:36:46.876292 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0809 09:36:46.876567 (+ 275us) server_negotiation.cc:299] Negotiation successful
0809 09:36:46.876673 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260809 09:36:46.877636 18280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:36:46.869450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:41497 (local address 127.20.125.130:60369)
0809 09:36:46.870162 (+ 712us) negotiation.cc:107] Waiting for socket to connect
0809 09:36:46.870206 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 09:36:46.871578 (+ 1372us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:36:46.872165 (+ 587us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:36:46.872172 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:36:46.872255 (+ 83us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:36:46.872723 (+ 468us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:36:46.872735 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:36:46.873552 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:36:46.873555 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:36:46.874432 (+ 877us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:36:46.874442 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:36:46.874983 (+ 541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:36:46.874987 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:36:46.875153 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:36:46.875833 (+ 680us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 09:36:46.875837 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:36:46.875893 (+ 56us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 09:36:46.875979 (+ 86us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 09:36:46.876367 (+ 388us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 09:36:46.876378 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 09:36:46.876384 (+ 6us) client_negotiation.cc:769] Sending connection context
0809 09:36:46.876514 (+ 130us) client_negotiation.cc:240] Negotiation successful
0809 09:36:46.876805 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":333,"thread_start_us":129,"threads_started":1}
I20260809 09:36:46.878727 20982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
/tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786267941944493-20982-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786267941944493-20982-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786267941944493-20982-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786267941944493-20982-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.125.131:0
--local_ip_for_outbound_sockets=127.20.125.131
--webserver_interface=127.20.125.131
--webserver_port=0
--tserver_master_addrs=127.20.125.190:41497
--builtin_ntp_servers=127.20.125.148:43133
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260809 09:36:46.879010 18278 heartbeater.cc:347] Connected to a master server at 127.20.125.190:41497
I20260809 09:36:46.879271 18278 heartbeater.cc:464] Registering TS with master...
I20260809 09:36:46.879784 18278 heartbeater.cc:511] Master 127.20.125.190:41497 requested a full tablet report, sending...
I20260809 09:36:46.880963 17803 ts_manager.cc:194] Registered new tserver with Master: 04beac347d7245508c817aa7ff612683 (127.20.125.130:46251)
I20260809 09:36:46.881527 17803 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.20.125.130:60369
W20260809 09:36:46.994782 18281 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 09:36:46.995064 18281 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 09:36:46.995134 18281 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 09:36:46.999218 18281 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 09:36:46.999293 18281 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 09:36:46.999377 18281 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.125.131
I20260809 09:36:47.004161 18281 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.125.148:43133
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786267941944493-20982-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786267941944493-20982-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.125.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786267941944493-20982-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.125.131
--webserver_port=0
--tserver_master_addrs=127.20.125.190:41497
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18281
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.125.131
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786267941944493-20982-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 08:38:55 UTC on ccycloud.quasar-njjleu.root.comops.site
build id 4575935
I20260809 09:36:47.005389 18281 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 09:36:47.006873 18281 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 09:36:47.016785 18286 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:36:47.016870 18287 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:36:47.016844 18281 server_base.cc:1034] running on GCE node
W20260809 09:36:47.017076 18289 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:36:47.017532 18281 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 09:36:47.018283 18281 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 09:36:47.019469 18281 hybrid_clock.cc:648] HybridClock initialized: now 1786268207019433 us; error 85 us; skew 500 ppm
I20260809 09:36:47.021438 18281 webserver.cc:460] Webserver started at http://127.20.125.131:35223/ using document root <none> and password file <none>
I20260809 09:36:47.022037 18281 fs_manager.cc:359] Metadata directory not provided
I20260809 09:36:47.022104 18281 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 09:36:47.022329 18281 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 09:36:47.024444 18281 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786267941944493-20982-0/minicluster-data/ts-2/data/instance:
uuid: "ef7ad916fa174b749d31029781013aa4"
format_stamp: "Formatted at 2026-08-09 09:36:47 on dist-test-slave-7kzw"
server_key: "3546d947395a2a14b91e114ee28bd09a"
server_key_iv: "925f2d74ebf6ef2a31e326b896091aa7"
server_key_version: "encryptionkey@0"
I20260809 09:36:47.025040 18281 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786267941944493-20982-0/minicluster-data/ts-2/wal/instance:
uuid: "ef7ad916fa174b749d31029781013aa4"
format_stamp: "Formatted at 2026-08-09 09:36:47 on dist-test-slave-7kzw"
server_key: "3546d947395a2a14b91e114ee28bd09a"
server_key_iv: "925f2d74ebf6ef2a31e326b896091aa7"
server_key_version: "encryptionkey@0"
I20260809 09:36:47.028841 18281 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.004s
I20260809 09:36:47.031337 18296 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 09:36:47.032563 18281 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 09:36:47.032707 18281 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786267941944493-20982-0/minicluster-data/ts-2/data,/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786267941944493-20982-0/minicluster-data/ts-2/wal
uuid: "ef7ad916fa174b749d31029781013aa4"
format_stamp: "Formatted at 2026-08-09 09:36:47 on dist-test-slave-7kzw"
server_key: "3546d947395a2a14b91e114ee28bd09a"
server_key_iv: "925f2d74ebf6ef2a31e326b896091aa7"
server_key_version: "encryptionkey@0"
I20260809 09:36:47.032823 18281 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786267941944493-20982-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786267941944493-20982-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786267941944493-20982-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 09:36:47.067677 18281 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 09:36:47.068454 18281 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 09:36:47.068681 18281 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 09:36:47.069335 18281 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 09:36:47.070343 18281 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 09:36:47.070416 18281 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 09:36:47.070505 18281 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 09:36:47.070560 18281 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 09:36:47.079664 18281 rpc_server.cc:307] RPC server started. Bound to: 127.20.125.131:33071
I20260809 09:36:47.079692 18470 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.125.131:33071 every 8 connection(s)
I20260809 09:36:47.080577 18281 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786267941944493-20982-0/minicluster-data/ts-2/data/info.pb
I20260809 09:36:47.087457 20982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu as pid 18281
I20260809 09:36:47.087661 20982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786267941944493-20982-0/minicluster-data/ts-2/wal/instance
I20260809 09:36:47.087963 20982 external_mini_cluster.cc:1428] Setting key 1f6cf36d1370003e93343b64c8a1fab0
I20260809 09:36:47.090420 17862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:36:47.082452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.131:42261 (local address 127.20.125.190:41497)
0809 09:36:47.082614 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0809 09:36:47.082618 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 09:36:47.083663 (+ 1045us) server_negotiation.cc:407] Connection header received
0809 09:36:47.085115 (+ 1452us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:36:47.085118 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:36:47.085176 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:36:47.085299 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:36:47.086197 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:36:47.086797 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:36:47.088060 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:36:47.088328 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:36:47.088453 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:36:47.089873 (+ 1420us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:36:47.089877 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:36:47.089879 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 09:36:47.089892 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:36:47.089915 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 09:36:47.089925 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:36:47.090011 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0809 09:36:47.090184 (+ 173us) server_negotiation.cc:299] Negotiation successful
0809 09:36:47.090245 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260809 09:36:47.091323 18475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:36:47.082855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:41497 (local address 127.20.125.131:42261)
0809 09:36:47.083511 (+ 656us) negotiation.cc:107] Waiting for socket to connect
0809 09:36:47.083552 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 09:36:47.084899 (+ 1347us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:36:47.085480 (+ 581us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:36:47.085487 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:36:47.085571 (+ 84us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:36:47.086052 (+ 481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:36:47.086064 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:36:47.086926 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:36:47.086929 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:36:47.087935 (+ 1006us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:36:47.087946 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:36:47.088525 (+ 579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:36:47.088530 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:36:47.088767 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:36:47.089522 (+ 755us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 09:36:47.089524 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:36:47.089582 (+ 58us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 09:36:47.089696 (+ 114us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 09:36:47.090020 (+ 324us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 09:36:47.090028 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 09:36:47.090033 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 09:36:47.090138 (+ 105us) client_negotiation.cc:240] Negotiation successful
0809 09:36:47.090447 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":267,"thread_start_us":116,"threads_started":1}
I20260809 09:36:47.092439 18473 heartbeater.cc:347] Connected to a master server at 127.20.125.190:41497
I20260809 09:36:47.092724 18473 heartbeater.cc:464] Registering TS with master...
I20260809 09:36:47.093250 18473 heartbeater.cc:511] Master 127.20.125.190:41497 requested a full tablet report, sending...
I20260809 09:36:47.094321 17803 ts_manager.cc:194] Registered new tserver with Master: ef7ad916fa174b749d31029781013aa4 (127.20.125.131:33071)
I20260809 09:36:47.094951 17803 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.20.125.131:42261
I20260809 09:36:47.102576 20982 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 09:36:47.108484 17862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:36:47.104714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51584 (local address 127.20.125.190:41497)
0809 09:36:47.104861 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0809 09:36:47.104865 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 09:36:47.105103 (+ 238us) server_negotiation.cc:407] Connection header received
0809 09:36:47.105187 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:36:47.105189 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:36:47.105229 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:36:47.105298 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:36:47.105583 (+ 285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:36:47.106086 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:36:47.106981 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:36:47.107213 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:36:47.107316 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:36:47.108107 (+ 791us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:36:47.108110 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:36:47.108112 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 09:36:47.108124 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:36:47.108144 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 09:36:47.108152 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:36:47.108214 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0809 09:36:47.108321 (+ 107us) server_negotiation.cc:299] Negotiation successful
0809 09:36:47.108371 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260809 09:36:47.111361 17803 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51584:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 09:36:47.114426 17803 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 09:36:47.129408 18487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:36:47.123646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.129:33511 (local address 127.0.0.1:40872)
0809 09:36:47.124269 (+ 623us) negotiation.cc:107] Waiting for socket to connect
0809 09:36:47.124280 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 09:36:47.124393 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:36:47.124926 (+ 533us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:36:47.124932 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:36:47.124949 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 09:36:47.125076 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:36:47.125083 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:36:47.126506 (+ 1423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:36:47.126509 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:36:47.127814 (+ 1305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:36:47.127824 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:36:47.128801 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:36:47.128804 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:36:47.128954 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:36:47.129014 (+ 60us) client_negotiation.cc:769] Sending connection context
0809 09:36:47.129136 (+ 122us) client_negotiation.cc:240] Negotiation successful
0809 09:36:47.129276 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":562,"mutex_wait_us":146,"thread_start_us":85,"threads_started":1}
I20260809 09:36:47.129491 18491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:36:47.123835 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40872 (local address 127.20.125.129:33511)
0809 09:36:47.124473 (+ 638us) server_negotiation.cc:206] Beginning negotiation
0809 09:36:47.124489 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 09:36:47.124514 (+ 25us) server_negotiation.cc:407] Connection header received
0809 09:36:47.124593 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:36:47.124598 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:36:47.124768 (+ 170us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:36:47.124935 (+ 167us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 09:36:47.125201 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:36:47.126326 (+ 1125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:36:47.127942 (+ 1616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:36:47.128664 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:36:47.128795 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:36:47.128983 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0809 09:36:47.129196 (+ 213us) server_negotiation.cc:299] Negotiation successful
0809 09:36:47.129306 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":538,"thread_start_us":105,"threads_started":1}
I20260809 09:36:47.130931 18486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:36:47.122874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.131:33071 (local address 127.0.0.1:41500)
0809 09:36:47.123227 (+ 353us) negotiation.cc:107] Waiting for socket to connect
0809 09:36:47.123275 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0809 09:36:47.123393 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:36:47.124606 (+ 1213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:36:47.124612 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:36:47.124635 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 09:36:47.124800 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:36:47.124809 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:36:47.126445 (+ 1636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:36:47.126448 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:36:47.128114 (+ 1666us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:36:47.128127 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:36:47.130355 (+ 2228us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:36:47.130360 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:36:47.130448 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:36:47.130667 (+ 219us) client_negotiation.cc:769] Sending connection context
0809 09:36:47.130743 (+ 76us) client_negotiation.cc:240] Negotiation successful
0809 09:36:47.130822 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":174,"mutex_wait_us":162,"thread_start_us":119,"threads_started":1}
I20260809 09:36:47.131269 18492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:36:47.123855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45100 (local address 127.20.125.130:46251)
0809 09:36:47.124648 (+ 793us) server_negotiation.cc:206] Beginning negotiation
0809 09:36:47.124657 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 09:36:47.124677 (+ 20us) server_negotiation.cc:407] Connection header received
0809 09:36:47.124739 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:36:47.124743 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:36:47.124916 (+ 173us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:36:47.125065 (+ 149us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 09:36:47.125437 (+ 372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:36:47.126175 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:36:47.128262 (+ 2087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:36:47.129056 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:36:47.129206 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:36:47.129365 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0809 09:36:47.131006 (+ 1641us) server_negotiation.cc:299] Negotiation successful
0809 09:36:47.131110 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":681,"thread_start_us":119,"threads_started":1}
I20260809 09:36:47.131538 18488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:36:47.123427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41500 (local address 127.20.125.131:33071)
0809 09:36:47.124058 (+ 631us) server_negotiation.cc:206] Beginning negotiation
0809 09:36:47.124077 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0809 09:36:47.124103 (+ 26us) server_negotiation.cc:407] Connection header received
0809 09:36:47.124195 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:36:47.124201 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:36:47.124423 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:36:47.124948 (+ 525us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 09:36:47.125091 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:36:47.126333 (+ 1242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:36:47.129435 (+ 3102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:36:47.130254 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:36:47.130377 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:36:47.130555 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0809 09:36:47.131242 (+ 687us) server_negotiation.cc:299] Negotiation successful
0809 09:36:47.131354 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":525,"thread_start_us":135,"threads_started":1}
I20260809 09:36:47.131541 18489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:36:47.122965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.130:46251 (local address 127.0.0.1:45100)
0809 09:36:47.123940 (+ 975us) negotiation.cc:107] Waiting for socket to connect
0809 09:36:47.123959 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 09:36:47.124051 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:36:47.125215 (+ 1164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:36:47.125224 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:36:47.125239 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 09:36:47.125315 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:36:47.125320 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:36:47.126324 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:36:47.126328 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:36:47.128114 (+ 1786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:36:47.128127 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:36:47.130627 (+ 2500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:36:47.130630 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:36:47.130755 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:36:47.130778 (+ 23us) client_negotiation.cc:769] Sending connection context
0809 09:36:47.130852 (+ 74us) client_negotiation.cc:240] Negotiation successful
0809 09:36:47.130922 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":907,"mutex_wait_us":214,"thread_start_us":118,"threads_started":1}
I20260809 09:36:47.132916 17987 tablet_service.cc:1467] Processing CreateTablet for tablet 692fe8442a3e49cfbe5ff495b7d4d7f3 (DEFAULT_TABLE table=test-table [id=ae36c2c86f90425fa3df0ed7b22fbfe2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 09:36:47.134035 17987 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 692fe8442a3e49cfbe5ff495b7d4d7f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 09:36:47.134169 18175 tablet_service.cc:1467] Processing CreateTablet for tablet 692fe8442a3e49cfbe5ff495b7d4d7f3 (DEFAULT_TABLE table=test-table [id=ae36c2c86f90425fa3df0ed7b22fbfe2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 09:36:47.135221 18175 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 692fe8442a3e49cfbe5ff495b7d4d7f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 09:36:47.134891 18379 tablet_service.cc:1467] Processing CreateTablet for tablet 692fe8442a3e49cfbe5ff495b7d4d7f3 (DEFAULT_TABLE table=test-table [id=ae36c2c86f90425fa3df0ed7b22fbfe2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 09:36:47.135842 18379 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 692fe8442a3e49cfbe5ff495b7d4d7f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 09:36:47.140769 18493 tablet_bootstrap.cc:492] T 692fe8442a3e49cfbe5ff495b7d4d7f3 P 586b0ffe5d5d42f7a5e36c74a3beb965: Bootstrap starting.
I20260809 09:36:47.143147 18493 tablet_bootstrap.cc:654] T 692fe8442a3e49cfbe5ff495b7d4d7f3 P 586b0ffe5d5d42f7a5e36c74a3beb965: Neither blocks nor log segments found. Creating new log.
I20260809 09:36:47.143356 18498 tablet_bootstrap.cc:492] T 692fe8442a3e49cfbe5ff495b7d4d7f3 P ef7ad916fa174b749d31029781013aa4: Bootstrap starting.
I20260809 09:36:47.143908 18494 tablet_bootstrap.cc:492] T 692fe8442a3e49cfbe5ff495b7d4d7f3 P 04beac347d7245508c817aa7ff612683: Bootstrap starting.
I20260809 09:36:47.144071 18493 log.cc:824] T 692fe8442a3e49cfbe5ff495b7d4d7f3 P 586b0ffe5d5d42f7a5e36c74a3beb965: Log is configured to *not* fsync() on all Append() calls
I20260809 09:36:47.145522 18498 tablet_bootstrap.cc:654] T 692fe8442a3e49cfbe5ff495b7d4d7f3 P ef7ad916fa174b749d31029781013aa4: Neither blocks nor log segments found. Creating new log.
I20260809 09:36:47.145907 18494 tablet_bootstrap.cc:654] T 692fe8442a3e49cfbe5ff495b7d4d7f3 P 04beac347d7245508c817aa7ff612683: Neither blocks nor log segments found. Creating new log.
I20260809 09:36:47.146487 18493 tablet_bootstrap.cc:492] T 692fe8442a3e49cfbe5ff495b7d4d7f3 P 586b0ffe5d5d42f7a5e36c74a3beb965: No bootstrap required, opened a new log
I20260809 09:36:47.146430 18498 log.cc:824] T 692fe8442a3e49cfbe5ff495b7d4d7f3 P ef7ad916fa174b749d31029781013aa4: Log is configured to *not* fsync() on all Append() calls
I20260809 09:36:47.146708 18494 log.cc:824] T 692fe8442a3e49cfbe5ff495b7d4d7f3 P 04beac347d7245508c817aa7ff612683: Log is configured to *not* fsync() on all Append() calls
I20260809 09:36:47.147439 18493 ts_tablet_manager.cc:1397] T 692fe8442a3e49cfbe5ff495b7d4d7f3 P 586b0ffe5d5d42f7a5e36c74a3beb965: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260809 09:36:47.148523 18498 tablet_bootstrap.cc:492] T 692fe8442a3e49cfbe5ff495b7d4d7f3 P ef7ad916fa174b749d31029781013aa4: No bootstrap required, opened a new log
I20260809 09:36:47.148696 18498 ts_tablet_manager.cc:1397] T 692fe8442a3e49cfbe5ff495b7d4d7f3 P ef7ad916fa174b749d31029781013aa4: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260809 09:36:47.149204 18494 tablet_bootstrap.cc:492] T 692fe8442a3e49cfbe5ff495b7d4d7f3 P 04beac347d7245508c817aa7ff612683: No bootstrap required, opened a new log
I20260809 09:36:47.149490 18494 ts_tablet_manager.cc:1397] T 692fe8442a3e49cfbe5ff495b7d4d7f3 P 04beac347d7245508c817aa7ff612683: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260809 09:36:47.152242 18493 raft_consensus.cc:348] T 692fe8442a3e49cfbe5ff495b7d4d7f3 P 586b0ffe5d5d42f7a5e36c74a3beb965 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef7ad916fa174b749d31029781013aa4" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 33071 } } peers { permanent_uuid: "04beac347d7245508c817aa7ff612683" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 46251 } } peers { permanent_uuid: "586b0ffe5d5d42f7a5e36c74a3beb965" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 33511 } }
I20260809 09:36:47.152567 18493 raft_consensus.cc:374] T 692fe8442a3e49cfbe5ff495b7d4d7f3 P 586b0ffe5d5d42f7a5e36c74a3beb965 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 09:36:47.152630 18493 raft_consensus.cc:729] T 692fe8442a3e49cfbe5ff495b7d4d7f3 P 586b0ffe5d5d42f7a5e36c74a3beb965 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 586b0ffe5d5d42f7a5e36c74a3beb965, State: Initialized, Role: FOLLOWER
I20260809 09:36:47.153091 18493 consensus_queue.cc:260] T 692fe8442a3e49cfbe5ff495b7d4d7f3 P 586b0ffe5d5d42f7a5e36c74a3beb965 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef7ad916fa174b749d31029781013aa4" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 33071 } } peers { permanent_uuid: "04beac347d7245508c817aa7ff612683" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 46251 } } peers { permanent_uuid: "586b0ffe5d5d42f7a5e36c74a3beb965" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 33511 } }
I20260809 09:36:47.153040 18498 raft_consensus.cc:348] T 692fe8442a3e49cfbe5ff495b7d4d7f3 P ef7ad916fa174b749d31029781013aa4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef7ad916fa174b749d31029781013aa4" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 33071 } } peers { permanent_uuid: "04beac347d7245508c817aa7ff612683" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 46251 } } peers { permanent_uuid: "586b0ffe5d5d42f7a5e36c74a3beb965" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 33511 } }
I20260809 09:36:47.153262 18498 raft_consensus.cc:374] T 692fe8442a3e49cfbe5ff495b7d4d7f3 P ef7ad916fa174b749d31029781013aa4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 09:36:47.153330 18498 raft_consensus.cc:729] T 692fe8442a3e49cfbe5ff495b7d4d7f3 P ef7ad916fa174b749d31029781013aa4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef7ad916fa174b749d31029781013aa4, State: Initialized, Role: FOLLOWER
I20260809 09:36:47.153651 18494 raft_consensus.cc:348] T 692fe8442a3e49cfbe5ff495b7d4d7f3 P 04beac347d7245508c817aa7ff612683 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef7ad916fa174b749d31029781013aa4" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 33071 } } peers { permanent_uuid: "04beac347d7245508c817aa7ff612683" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 46251 } } peers { permanent_uuid: "586b0ffe5d5d42f7a5e36c74a3beb965" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 33511 } }
I20260809 09:36:47.153796 18498 consensus_queue.cc:260] T 692fe8442a3e49cfbe5ff495b7d4d7f3 P ef7ad916fa174b749d31029781013aa4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef7ad916fa174b749d31029781013aa4" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 33071 } } peers { permanent_uuid: "04beac347d7245508c817aa7ff612683" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 46251 } } peers { permanent_uuid: "586b0ffe5d5d42f7a5e36c74a3beb965" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 33511 } }
I20260809 09:36:47.153892 18494 raft_consensus.cc:374] T 692fe8442a3e49cfbe5ff495b7d4d7f3 P 04beac347d7245508c817aa7ff612683 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 09:36:47.153964 18494 raft_consensus.cc:729] T 692fe8442a3e49cfbe5ff495b7d4d7f3 P 04beac347d7245508c817aa7ff612683 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 04beac347d7245508c817aa7ff612683, State: Initialized, Role: FOLLOWER
I20260809 09:36:47.154157 18493 ts_tablet_manager.cc:1428] T 692fe8442a3e49cfbe5ff495b7d4d7f3 P 586b0ffe5d5d42f7a5e36c74a3beb965: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260809 09:36:47.154364 18494 consensus_queue.cc:260] T 692fe8442a3e49cfbe5ff495b7d4d7f3 P 04beac347d7245508c817aa7ff612683 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef7ad916fa174b749d31029781013aa4" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 33071 } } peers { permanent_uuid: "04beac347d7245508c817aa7ff612683" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 46251 } } peers { permanent_uuid: "586b0ffe5d5d42f7a5e36c74a3beb965" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 33511 } }
I20260809 09:36:47.154520 18473 heartbeater.cc:503] Master 127.20.125.190:41497 was elected leader, sending a full tablet report...
I20260809 09:36:47.154527 18081 heartbeater.cc:503] Master 127.20.125.190:41497 was elected leader, sending a full tablet report...
I20260809 09:36:47.154814 18498 ts_tablet_manager.cc:1428] T 692fe8442a3e49cfbe5ff495b7d4d7f3 P ef7ad916fa174b749d31029781013aa4: Time spent starting tablet: real 0.006s user 0.002s sys 0.005s
I20260809 09:36:47.155280 18494 ts_tablet_manager.cc:1428] T 692fe8442a3e49cfbe5ff495b7d4d7f3 P 04beac347d7245508c817aa7ff612683: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260809 09:36:47.155299 18278 heartbeater.cc:503] Master 127.20.125.190:41497 was elected leader, sending a full tablet report...
I20260809 09:36:47.167601 18505 raft_consensus.cc:482] T 692fe8442a3e49cfbe5ff495b7d4d7f3 P 04beac347d7245508c817aa7ff612683 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 09:36:47.167791 18505 raft_consensus.cc:504] T 692fe8442a3e49cfbe5ff495b7d4d7f3 P 04beac347d7245508c817aa7ff612683 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef7ad916fa174b749d31029781013aa4" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 33071 } } peers { permanent_uuid: "04beac347d7245508c817aa7ff612683" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 46251 } } peers { permanent_uuid: "586b0ffe5d5d42f7a5e36c74a3beb965" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 33511 } }
I20260809 09:36:47.168759 18505 leader_election.cc:288] T 692fe8442a3e49cfbe5ff495b7d4d7f3 P 04beac347d7245508c817aa7ff612683 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ef7ad916fa174b749d31029781013aa4 (127.20.125.131:33071), 586b0ffe5d5d42f7a5e36c74a3beb965 (127.20.125.129:33511)
I20260809 09:36:47.172689 18491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:36:47.169144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.130:52727 (local address 127.20.125.129:33511)
0809 09:36:47.169276 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0809 09:36:47.169280 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 09:36:47.169296 (+ 16us) server_negotiation.cc:407] Connection header received
0809 09:36:47.169431 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:36:47.169436 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:36:47.169499 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:36:47.169577 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 09:36:47.169913 (+ 336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:36:47.170657 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:36:47.171694 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:36:47.172078 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:36:47.172206 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:36:47.172266 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0809 09:36:47.172491 (+ 225us) server_negotiation.cc:299] Negotiation successful
0809 09:36:47.172559 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260809 09:36:47.172712 18280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:36:47.168985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.129:33511 (local address 127.20.125.130:52727)
0809 09:36:47.169192 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0809 09:36:47.169208 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 09:36:47.169323 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:36:47.169620 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:36:47.169624 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:36:47.169643 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 09:36:47.169777 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:36:47.169787 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:36:47.170776 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:36:47.170780 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:36:47.171560 (+ 780us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:36:47.171569 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:36:47.172207 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:36:47.172210 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:36:47.172358 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:36:47.172381 (+ 23us) client_negotiation.cc:769] Sending connection context
0809 09:36:47.172471 (+ 90us) client_negotiation.cc:240] Negotiation successful
0809 09:36:47.172544 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"mutex_wait_us":48}
I20260809 09:36:47.172722 18488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:36:47.169068 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.125.130:56103 (local address 127.20.125.131:33071)
0809 09:36:47.169191 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0809 09:36:47.169196 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 09:36:47.169523 (+ 327us) server_negotiation.cc:407] Connection header received
0809 09:36:47.169655 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:36:47.169658 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:36:47.169700 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:36:47.169792 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 09:36:47.170091 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:36:47.170766 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:36:47.171756 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:36:47.172148 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:36:47.172293 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:36:47.172341 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0809 09:36:47.172525 (+ 184us) server_negotiation.cc:299] Negotiation successful
0809 09:36:47.172590 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260809 09:36:47.172739 18506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:36:47.168985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.131:33071 (local address 127.20.125.130:56103)
0809 09:36:47.169416 (+ 431us) negotiation.cc:107] Waiting for socket to connect
0809 09:36:47.169430 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 09:36:47.169540 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:36:47.169819 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:36:47.169822 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:36:47.169840 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 09:36:47.169976 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:36:47.169982 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:36:47.170879 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:36:47.170884 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:36:47.171635 (+ 751us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:36:47.171644 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:36:47.172271 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:36:47.172275 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:36:47.172397 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:36:47.172422 (+ 25us) client_negotiation.cc:769] Sending connection context
0809 09:36:47.172511 (+ 89us) client_negotiation.cc:240] Negotiation successful
0809 09:36:47.172577 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":341,"mutex_wait_us":37,"thread_start_us":122,"threads_started":1}
I20260809 09:36:47.173424 18405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "692fe8442a3e49cfbe5ff495b7d4d7f3" candidate_uuid: "04beac347d7245508c817aa7ff612683" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef7ad916fa174b749d31029781013aa4" is_pre_election: true
I20260809 09:36:47.173429 18016 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "692fe8442a3e49cfbe5ff495b7d4d7f3" candidate_uuid: "04beac347d7245508c817aa7ff612683" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "586b0ffe5d5d42f7a5e36c74a3beb965" is_pre_election: true
I20260809 09:36:47.173713 18016 raft_consensus.cc:2454] T 692fe8442a3e49cfbe5ff495b7d4d7f3 P 586b0ffe5d5d42f7a5e36c74a3beb965 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 04beac347d7245508c817aa7ff612683 in term 0.
I20260809 09:36:47.173713 18405 raft_consensus.cc:2454] T 692fe8442a3e49cfbe5ff495b7d4d7f3 P ef7ad916fa174b749d31029781013aa4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 04beac347d7245508c817aa7ff612683 in term 0.
I20260809 09:36:47.174221 18111 leader_election.cc:302] T 692fe8442a3e49cfbe5ff495b7d4d7f3 P 04beac347d7245508c817aa7ff612683 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 04beac347d7245508c817aa7ff612683, 586b0ffe5d5d42f7a5e36c74a3beb965, ef7ad916fa174b749d31029781013aa4; no voters:
I20260809 09:36:47.174520 18505 raft_consensus.cc:2781] T 692fe8442a3e49cfbe5ff495b7d4d7f3 P 04beac347d7245508c817aa7ff612683 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 09:36:47.174611 18505 raft_consensus.cc:482] T 692fe8442a3e49cfbe5ff495b7d4d7f3 P 04beac347d7245508c817aa7ff612683 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 09:36:47.174664 18505 raft_consensus.cc:3037] T 692fe8442a3e49cfbe5ff495b7d4d7f3 P 04beac347d7245508c817aa7ff612683 [term 0 FOLLOWER]: Advancing to term 1
I20260809 09:36:47.175724 18505 raft_consensus.cc:504] T 692fe8442a3e49cfbe5ff495b7d4d7f3 P 04beac347d7245508c817aa7ff612683 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef7ad916fa174b749d31029781013aa4" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 33071 } } peers { permanent_uuid: "04beac347d7245508c817aa7ff612683" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 46251 } } peers { permanent_uuid: "586b0ffe5d5d42f7a5e36c74a3beb965" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 33511 } }
I20260809 09:36:47.176087 18505 leader_election.cc:288] T 692fe8442a3e49cfbe5ff495b7d4d7f3 P 04beac347d7245508c817aa7ff612683 [CANDIDATE]: Term 1 election: Requested vote from peers ef7ad916fa174b749d31029781013aa4 (127.20.125.131:33071), 586b0ffe5d5d42f7a5e36c74a3beb965 (127.20.125.129:33511)
I20260809 09:36:47.176440 18016 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "692fe8442a3e49cfbe5ff495b7d4d7f3" candidate_uuid: "04beac347d7245508c817aa7ff612683" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "586b0ffe5d5d42f7a5e36c74a3beb965"
I20260809 09:36:47.176440 18405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "692fe8442a3e49cfbe5ff495b7d4d7f3" candidate_uuid: "04beac347d7245508c817aa7ff612683" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef7ad916fa174b749d31029781013aa4"
I20260809 09:36:47.176558 18016 raft_consensus.cc:3037] T 692fe8442a3e49cfbe5ff495b7d4d7f3 P 586b0ffe5d5d42f7a5e36c74a3beb965 [term 0 FOLLOWER]: Advancing to term 1
I20260809 09:36:47.176558 18405 raft_consensus.cc:3037] T 692fe8442a3e49cfbe5ff495b7d4d7f3 P ef7ad916fa174b749d31029781013aa4 [term 0 FOLLOWER]: Advancing to term 1
W20260809 09:36:47.176896 18083 tablet.cc:2369] T 692fe8442a3e49cfbe5ff495b7d4d7f3 P 586b0ffe5d5d42f7a5e36c74a3beb965: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 09:36:47.177803 18405 raft_consensus.cc:2454] T 692fe8442a3e49cfbe5ff495b7d4d7f3 P ef7ad916fa174b749d31029781013aa4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 04beac347d7245508c817aa7ff612683 in term 1.
I20260809 09:36:47.177803 18016 raft_consensus.cc:2454] T 692fe8442a3e49cfbe5ff495b7d4d7f3 P 586b0ffe5d5d42f7a5e36c74a3beb965 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 04beac347d7245508c817aa7ff612683 in term 1.
I20260809 09:36:47.178128 18109 leader_election.cc:302] T 692fe8442a3e49cfbe5ff495b7d4d7f3 P 04beac347d7245508c817aa7ff612683 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 04beac347d7245508c817aa7ff612683, 586b0ffe5d5d42f7a5e36c74a3beb965; no voters:
I20260809 09:36:47.178333 18505 raft_consensus.cc:2781] T 692fe8442a3e49cfbe5ff495b7d4d7f3 P 04beac347d7245508c817aa7ff612683 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 09:36:47.178586 18505 raft_consensus.cc:686] T 692fe8442a3e49cfbe5ff495b7d4d7f3 P 04beac347d7245508c817aa7ff612683 [term 1 LEADER]: Becoming Leader. State: Replica: 04beac347d7245508c817aa7ff612683, State: Running, Role: LEADER
I20260809 09:36:47.178933 18505 consensus_queue.cc:237] T 692fe8442a3e49cfbe5ff495b7d4d7f3 P 04beac347d7245508c817aa7ff612683 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef7ad916fa174b749d31029781013aa4" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 33071 } } peers { permanent_uuid: "04beac347d7245508c817aa7ff612683" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 46251 } } peers { permanent_uuid: "586b0ffe5d5d42f7a5e36c74a3beb965" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 33511 } }
I20260809 09:36:47.182189 17804 catalog_manager.cc:5495] T 692fe8442a3e49cfbe5ff495b7d4d7f3 P 04beac347d7245508c817aa7ff612683 reported cstate change: term changed from 0 to 1, leader changed from <none> to 04beac347d7245508c817aa7ff612683 (127.20.125.130). New cstate: current_term: 1 leader_uuid: "04beac347d7245508c817aa7ff612683" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef7ad916fa174b749d31029781013aa4" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 33071 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "04beac347d7245508c817aa7ff612683" member_type: VOTER last_known_addr { host: "127.20.125.130" port: 46251 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "586b0ffe5d5d42f7a5e36c74a3beb965" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 33511 } health_report { overall_health: UNKNOWN } } }
I20260809 09:36:47.207926 18492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:36:47.203912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45108 (local address 127.20.125.130:46251)
0809 09:36:47.204094 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0809 09:36:47.204098 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 09:36:47.204114 (+ 16us) server_negotiation.cc:407] Connection header received
0809 09:36:47.204248 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:36:47.204250 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:36:47.204362 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:36:47.204454 (+ 92us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 09:36:47.204814 (+ 360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:36:47.205372 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:36:47.206404 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:36:47.206639 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:36:47.206749 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:36:47.207048 (+ 299us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 09:36:47.207476 (+ 428us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 09:36:47.207565 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0809 09:36:47.207760 (+ 195us) server_negotiation.cc:299] Negotiation successful
0809 09:36:47.207809 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260809 09:36:47.214039 18016 raft_consensus.cc:1261] T 692fe8442a3e49cfbe5ff495b7d4d7f3 P 586b0ffe5d5d42f7a5e36c74a3beb965 [term 1 FOLLOWER]: Refusing update from remote peer 04beac347d7245508c817aa7ff612683: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 09:36:47.214035 18405 raft_consensus.cc:1261] T 692fe8442a3e49cfbe5ff495b7d4d7f3 P ef7ad916fa174b749d31029781013aa4 [term 1 FOLLOWER]: Refusing update from remote peer 04beac347d7245508c817aa7ff612683: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 09:36:47.214775 18507 consensus_queue.cc:1035] T 692fe8442a3e49cfbe5ff495b7d4d7f3 P 04beac347d7245508c817aa7ff612683 [LEADER]: Connected to new peer: Peer: permanent_uuid: "586b0ffe5d5d42f7a5e36c74a3beb965" member_type: VOTER last_known_addr { host: "127.20.125.129" port: 33511 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 09:36:47.214977 18505 consensus_queue.cc:1035] T 692fe8442a3e49cfbe5ff495b7d4d7f3 P 04beac347d7245508c817aa7ff612683 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef7ad916fa174b749d31029781013aa4" member_type: VOTER last_known_addr { host: "127.20.125.131" port: 33071 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 09:36:47.224136 18516 mvcc.cc:205] Tried to move back new op lower bound from 7316554576740888576 to 7316554576607649792. Current Snapshot: MvccSnapshot[applied={T|T < 7316554576740888576}]
I20260809 09:36:47.231056 17803 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:51584:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 09:36:47.231254 17803 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:51584:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 09:36:47.233768 17803 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8958145386e244499f9adca3756f17c8: Sending DeleteTablet for 3 replicas of tablet 692fe8442a3e49cfbe5ff495b7d4d7f3
I20260809 09:36:47.234583 18379 tablet_service.cc:1514] Processing DeleteTablet for tablet 692fe8442a3e49cfbe5ff495b7d4d7f3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 09:36:47 UTC) from {username='slave'} at 127.0.0.1:41500
I20260809 09:36:47.234877 18175 tablet_service.cc:1514] Processing DeleteTablet for tablet 692fe8442a3e49cfbe5ff495b7d4d7f3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 09:36:47 UTC) from {username='slave'} at 127.0.0.1:45100
I20260809 09:36:47.235180 17987 tablet_service.cc:1514] Processing DeleteTablet for tablet 692fe8442a3e49cfbe5ff495b7d4d7f3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 09:36:47 UTC) from {username='slave'} at 127.0.0.1:40872
I20260809 09:36:47.235420 18528 tablet_replica.cc:330] T 692fe8442a3e49cfbe5ff495b7d4d7f3 P 04beac347d7245508c817aa7ff612683: stopping tablet replica
I20260809 09:36:47.235481 18527 tablet_replica.cc:330] T 692fe8442a3e49cfbe5ff495b7d4d7f3 P ef7ad916fa174b749d31029781013aa4: stopping tablet replica
I20260809 09:36:47.235743 18528 raft_consensus.cc:2229] T 692fe8442a3e49cfbe5ff495b7d4d7f3 P 04beac347d7245508c817aa7ff612683 [term 1 LEADER]: Raft consensus shutting down.
I20260809 09:36:47.236107 20982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu with pid 17890
I20260809 09:36:47.240936 18527 raft_consensus.cc:2229] T 692fe8442a3e49cfbe5ff495b7d4d7f3 P ef7ad916fa174b749d31029781013aa4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 09:36:47.241374 18527 raft_consensus.cc:2258] T 692fe8442a3e49cfbe5ff495b7d4d7f3 P ef7ad916fa174b749d31029781013aa4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 09:36:47.243335 18528 raft_consensus.cc:2258] T 692fe8442a3e49cfbe5ff495b7d4d7f3 P 04beac347d7245508c817aa7ff612683 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 09:36:47.243423 18527 ts_tablet_manager.cc:1905] T 692fe8442a3e49cfbe5ff495b7d4d7f3 P ef7ad916fa174b749d31029781013aa4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 09:36:47.245714 18528 ts_tablet_manager.cc:1905] T 692fe8442a3e49cfbe5ff495b7d4d7f3 P 04beac347d7245508c817aa7ff612683: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 09:36:47.246855 18527 ts_tablet_manager.cc:1918] T 692fe8442a3e49cfbe5ff495b7d4d7f3 P ef7ad916fa174b749d31029781013aa4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 09:36:47.246984 18527 log.cc:1196] T 692fe8442a3e49cfbe5ff495b7d4d7f3 P ef7ad916fa174b749d31029781013aa4: Deleting WAL directory at /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786267941944493-20982-0/minicluster-data/ts-2/wal/wals/692fe8442a3e49cfbe5ff495b7d4d7f3
I20260809 09:36:47.247426 18527 ts_tablet_manager.cc:1939] T 692fe8442a3e49cfbe5ff495b7d4d7f3 P ef7ad916fa174b749d31029781013aa4: Deleting consensus metadata
I20260809 09:36:47.249029 17785 catalog_manager.cc:4841] TS ef7ad916fa174b749d31029781013aa4 (127.20.125.131:33071): tablet 692fe8442a3e49cfbe5ff495b7d4d7f3 (table test-table [id=ae36c2c86f90425fa3df0ed7b22fbfe2]) successfully deleted
I20260809 09:36:47.249152 18528 ts_tablet_manager.cc:1918] T 692fe8442a3e49cfbe5ff495b7d4d7f3 P 04beac347d7245508c817aa7ff612683: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 09:36:47.249253 18528 log.cc:1196] T 692fe8442a3e49cfbe5ff495b7d4d7f3 P 04beac347d7245508c817aa7ff612683: Deleting WAL directory at /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786267941944493-20982-0/minicluster-data/ts-1/wal/wals/692fe8442a3e49cfbe5ff495b7d4d7f3
I20260809 09:36:47.249591 18528 ts_tablet_manager.cc:1939] T 692fe8442a3e49cfbe5ff495b7d4d7f3 P 04beac347d7245508c817aa7ff612683: Deleting consensus metadata
I20260809 09:36:47.250819 17783 catalog_manager.cc:4841] TS 04beac347d7245508c817aa7ff612683 (127.20.125.130:46251): tablet 692fe8442a3e49cfbe5ff495b7d4d7f3 (table test-table [id=ae36c2c86f90425fa3df0ed7b22fbfe2]) successfully deleted
W20260809 09:36:47.253484 17783 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.20.125.129:33511 (error 108)
I20260809 09:36:47.254446 20982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu with pid 18087
I20260809 09:36:47.254760 18489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:36:47.254365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.129:33511 (local address 127.0.0.1:40884)
0809 09:36:47.254561 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0809 09:36:47.254652 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.129:33511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
W20260809 09:36:47.263842 17783 catalog_manager.cc:4584] TS 586b0ffe5d5d42f7a5e36c74a3beb965 (127.20.125.129:33511): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 692fe8442a3e49cfbe5ff495b7d4d7f3: Network error: Client connection negotiation failed: client connection to 127.20.125.129:33511: connect: Connection refused (error 111)
I20260809 09:36:47.270018 20982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu with pid 18281
I20260809 09:36:47.283015 20982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu with pid 17755
2026-08-09T09:36:47Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (1080 ms)
[----------] 6 tests from EncryptionPolicyTest (11117 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786267941944493-20982-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 09:36:47 dist-test-slave-7kzw krb5kdc[18534](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 09:36:47 dist-test-slave-7kzw krb5kdc[18534](info): set up 2 sockets
Aug 09 09:36:47 dist-test-slave-7kzw krb5kdc[18534](info): commencing operation
krb5kdc: starting...
W20260809 09:36:48.529561 20982 mini_kdc.cc:121] Time spent starting KDC: real 1.220s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786267941944493-20982-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786267941944493-20982-0/krb5kdc/test-user.keytab.
Aug 09 09:36:48 dist-test-slave-7kzw krb5kdc[18534](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268208, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T09:36:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T09:36:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786267941944493-20982-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786267941944493-20982-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 09:36:48.665212 20982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
/tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786267941944493-20982-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786267941944493-20982-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786267941944493-20982-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786267941944493-20982-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33041
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786267941944493-20982-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37903
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33041
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786267941944493-20982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786267941944493-20982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786267941944493-20982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 09:36:48.777881 18554 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 09:36:48.778143 18554 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 09:36:48.778185 18554 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 09:36:48.785324 18554 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 09:36:48.785427 18554 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 09:36:48.785460 18554 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 09:36:48.785488 18554 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 09:36:48.785528 18554 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 09:36:48.792886 18554 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37903
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786267941944493-20982-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786267941944493-20982-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33041
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786267941944493-20982-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33041
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786267941944493-20982-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18554
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786267941944493-20982-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 08:38:55 UTC on ccycloud.quasar-njjleu.root.comops.site
build id 4575935
I20260809 09:36:48.794232 18554 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 09:36:48.795537 18554 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 09:36:48.805158 18563 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:36:48.805382 18554 server_base.cc:1034] running on GCE node
W20260809 09:36:48.805179 18560 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:36:48.805135 18561 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:36:48.806174 18554 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 09:36:48.807457 18554 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 09:36:48.808704 18554 hybrid_clock.cc:648] HybridClock initialized: now 1786268208808678 us; error 45 us; skew 500 ppm
Aug 09 09:36:48 dist-test-slave-7kzw krb5kdc[18534](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268208, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 09:36:48.811898 18554 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 09:36:48.812827 18554 webserver.cc:460] Webserver started at http://127.0.0.1:44411/ using document root <none> and password file <none>
I20260809 09:36:48.813380 18554 fs_manager.cc:359] Metadata directory not provided
I20260809 09:36:48.813443 18554 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 09:36:48.813632 18554 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 09:36:48.815850 18554 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786267941944493-20982-0/minicluster-data/master-0/data/instance:
uuid: "cf29a1f0222440ff9d3b52dc519516ee"
format_stamp: "Formatted at 2026-08-09 09:36:48 on dist-test-slave-7kzw"
server_key: "192dc32afe11ae466d231709cee3d265"
server_key_iv: "2119b3521643bfb98de61ea7b013d019"
server_key_version: "encryptionkey@0"
I20260809 09:36:48.816391 18554 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786267941944493-20982-0/minicluster-data/master-0/wal/instance:
uuid: "cf29a1f0222440ff9d3b52dc519516ee"
format_stamp: "Formatted at 2026-08-09 09:36:48 on dist-test-slave-7kzw"
server_key: "192dc32afe11ae466d231709cee3d265"
server_key_iv: "2119b3521643bfb98de61ea7b013d019"
server_key_version: "encryptionkey@0"
I20260809 09:36:48.820394 18554 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 09:36:48.823235 18571 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 09:36:48.824693 18554 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 09:36:48.824811 18554 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786267941944493-20982-0/minicluster-data/master-0/data,/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786267941944493-20982-0/minicluster-data/master-0/wal
uuid: "cf29a1f0222440ff9d3b52dc519516ee"
format_stamp: "Formatted at 2026-08-09 09:36:48 on dist-test-slave-7kzw"
server_key: "192dc32afe11ae466d231709cee3d265"
server_key_iv: "2119b3521643bfb98de61ea7b013d019"
server_key_version: "encryptionkey@0"
I20260809 09:36:48.824913 18554 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786267941944493-20982-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786267941944493-20982-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786267941944493-20982-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 09:36:48.840903 18554 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 09:36:48.844173 18554 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 09:36:48.844347 18554 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 09:36:48.852017 18554 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33041
I20260809 09:36:48.852070 18658 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33041 every 8 connection(s)
I20260809 09:36:48.853108 18554 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786267941944493-20982-0/minicluster-data/master-0/data/info.pb
I20260809 09:36:48.855993 18659 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 09:36:48.861433 18659 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cf29a1f0222440ff9d3b52dc519516ee: Bootstrap starting.
I20260809 09:36:48.862659 20982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu as pid 18554
I20260809 09:36:48.862816 20982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786267941944493-20982-0/minicluster-data/master-0/wal/instance
I20260809 09:36:48.863168 20982 external_mini_cluster.cc:1428] Setting key 3307e900d43b846c47093d23e4c9f84f
I20260809 09:36:48.864092 18659 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cf29a1f0222440ff9d3b52dc519516ee: Neither blocks nor log segments found. Creating new log.
I20260809 09:36:48.864959 18659 log.cc:824] T 00000000000000000000000000000000 P cf29a1f0222440ff9d3b52dc519516ee: Log is configured to *not* fsync() on all Append() calls
I20260809 09:36:48.866909 18659 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cf29a1f0222440ff9d3b52dc519516ee: No bootstrap required, opened a new log
I20260809 09:36:48.870326 18659 raft_consensus.cc:348] T 00000000000000000000000000000000 P cf29a1f0222440ff9d3b52dc519516ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf29a1f0222440ff9d3b52dc519516ee" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33041 } }
I20260809 09:36:48.870649 18659 raft_consensus.cc:374] T 00000000000000000000000000000000 P cf29a1f0222440ff9d3b52dc519516ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 09:36:48.870743 18659 raft_consensus.cc:729] T 00000000000000000000000000000000 P cf29a1f0222440ff9d3b52dc519516ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf29a1f0222440ff9d3b52dc519516ee, State: Initialized, Role: FOLLOWER
I20260809 09:36:48.871222 18659 consensus_queue.cc:260] T 00000000000000000000000000000000 P cf29a1f0222440ff9d3b52dc519516ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf29a1f0222440ff9d3b52dc519516ee" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33041 } }
I20260809 09:36:48.871379 18659 raft_consensus.cc:388] T 00000000000000000000000000000000 P cf29a1f0222440ff9d3b52dc519516ee [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 09:36:48.871447 18659 raft_consensus.cc:482] T 00000000000000000000000000000000 P cf29a1f0222440ff9d3b52dc519516ee [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 09:36:48.871551 18659 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cf29a1f0222440ff9d3b52dc519516ee [term 0 FOLLOWER]: Advancing to term 1
Aug 09 09:36:48 dist-test-slave-7kzw krb5kdc[18534](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268208, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260809 09:36:48.872696 18659 raft_consensus.cc:504] T 00000000000000000000000000000000 P cf29a1f0222440ff9d3b52dc519516ee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf29a1f0222440ff9d3b52dc519516ee" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33041 } }
I20260809 09:36:48.873050 18659 leader_election.cc:302] T 00000000000000000000000000000000 P cf29a1f0222440ff9d3b52dc519516ee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cf29a1f0222440ff9d3b52dc519516ee; no voters:
I20260809 09:36:48.873397 18659 leader_election.cc:288] T 00000000000000000000000000000000 P cf29a1f0222440ff9d3b52dc519516ee [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 09:36:48.873520 18669 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cf29a1f0222440ff9d3b52dc519516ee [term 1 FOLLOWER]: Leader election won for term 1
I20260809 09:36:48.873754 18669 raft_consensus.cc:686] T 00000000000000000000000000000000 P cf29a1f0222440ff9d3b52dc519516ee [term 1 LEADER]: Becoming Leader. State: Replica: cf29a1f0222440ff9d3b52dc519516ee, State: Running, Role: LEADER
I20260809 09:36:48.874069 18669 consensus_queue.cc:237] T 00000000000000000000000000000000 P cf29a1f0222440ff9d3b52dc519516ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf29a1f0222440ff9d3b52dc519516ee" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33041 } }
I20260809 09:36:48.874787 18659 sys_catalog.cc:564] T 00000000000000000000000000000000 P cf29a1f0222440ff9d3b52dc519516ee [sys.catalog]: configured and running, proceeding with master startup.
I20260809 09:36:48.875903 18670 sys_catalog.cc:455] T 00000000000000000000000000000000 P cf29a1f0222440ff9d3b52dc519516ee [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cf29a1f0222440ff9d3b52dc519516ee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf29a1f0222440ff9d3b52dc519516ee" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33041 } } }
I20260809 09:36:48.876044 18670 sys_catalog.cc:458] T 00000000000000000000000000000000 P cf29a1f0222440ff9d3b52dc519516ee [sys.catalog]: This master's current role is: LEADER
I20260809 09:36:48.876323 18671 sys_catalog.cc:455] T 00000000000000000000000000000000 P cf29a1f0222440ff9d3b52dc519516ee [sys.catalog]: SysCatalogTable state changed. Reason: New leader cf29a1f0222440ff9d3b52dc519516ee. Latest consensus state: current_term: 1 leader_uuid: "cf29a1f0222440ff9d3b52dc519516ee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf29a1f0222440ff9d3b52dc519516ee" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33041 } } }
I20260809 09:36:48.876422 18671 sys_catalog.cc:458] T 00000000000000000000000000000000 P cf29a1f0222440ff9d3b52dc519516ee [sys.catalog]: This master's current role is: LEADER
I20260809 09:36:48.876721 18680 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 09:36:48.879021 18665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:36:48.864405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51597 (local address 127.0.0.1:33041)
0809 09:36:48.864888 (+ 483us) server_negotiation.cc:206] Beginning negotiation
0809 09:36:48.864905 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0809 09:36:48.864942 (+ 37us) server_negotiation.cc:407] Connection header received
0809 09:36:48.865646 (+ 704us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:36:48.865674 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:36:48.865997 (+ 323us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:36:48.866351 (+ 354us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:36:48.867296 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:36:48.868265 (+ 969us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:36:48.869139 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:36:48.869628 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:36:48.869814 (+ 186us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:36:48.872856 (+ 3042us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:36:48.872863 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:36:48.872880 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:36:48.872915 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:36:48.875246 (+ 2331us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:36:48.875873 (+ 627us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:36:48.875881 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:36:48.875889 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:36:48.876105 (+ 216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:36:48.876475 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:36:48.876481 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:36:48.876484 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:36:48.877024 (+ 540us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:36:48.877257 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0809 09:36:48.877776 (+ 519us) server_negotiation.cc:299] Negotiation successful
0809 09:36:48.878026 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":327,"thread_start_us":136,"threads_started":1}
I20260809 09:36:48.879949 18680 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 09:36:48.881947 20982 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 09:36:48.886087 18680 catalog_manager.cc:1347] Generated new cluster ID: cc3db1dab90343c8bc1d0dea0a009b53
I20260809 09:36:48.886176 18680 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 09:36:48.899575 18665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:36:48.887118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33069 (local address 127.0.0.1:33041)
0809 09:36:48.890639 (+ 3521us) server_negotiation.cc:206] Beginning negotiation
0809 09:36:48.890644 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 09:36:48.890659 (+ 15us) server_negotiation.cc:407] Connection header received
0809 09:36:48.890702 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:36:48.890704 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:36:48.890747 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:36:48.890826 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:36:48.891735 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:36:48.892514 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:36:48.893675 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:36:48.894014 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:36:48.894103 (+ 89us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:36:48.895864 (+ 1761us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:36:48.895868 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:36:48.895870 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:36:48.895895 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:36:48.897633 (+ 1738us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:36:48.898180 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:36:48.898183 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:36:48.898185 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:36:48.898265 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:36:48.898643 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:36:48.898646 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:36:48.898648 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:36:48.898854 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:36:48.898993 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0809 09:36:48.899300 (+ 307us) server_negotiation.cc:299] Negotiation successful
0809 09:36:48.899404 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3443}
I20260809 09:36:48.901983 18680 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 09:36:48.903227 18680 catalog_manager.cc:1504] Loading token signing keys...
I20260809 09:36:48.911440 18680 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cf29a1f0222440ff9d3b52dc519516ee: Generated new TSK 0
I20260809 09:36:48.912298 18680 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 09:36:48.918061 20982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu with pid 18554
W20260809 09:36:48.928934 18532 connection.cc:535] client connection to 127.0.0.1:33041 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:33041): Connection reset by peer (error 104)
2026-08-09T09:36:48Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1633 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786267941944493-20982-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 09:36:48 dist-test-slave-7kzw krb5kdc[18700](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 09:36:48 dist-test-slave-7kzw krb5kdc[18700](info): set up 2 sockets
Aug 09 09:36:48 dist-test-slave-7kzw krb5kdc[18700](info): commencing operation
krb5kdc: starting...
W20260809 09:36:50.169121 20982 mini_kdc.cc:121] Time spent starting KDC: real 1.225s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786267941944493-20982-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786267941944493-20982-0/krb5kdc/test-user.keytab.
Aug 09 09:36:50 dist-test-slave-7kzw krb5kdc[18700](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268210, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T09:36:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T09:36:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786267941944493-20982-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786267941944493-20982-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 09:36:50.309985 20982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
/tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786267941944493-20982-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786267941944493-20982-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786267941944493-20982-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786267941944493-20982-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37015
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786267941944493-20982-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37297
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37015
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786267941944493-20982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786267941944493-20982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786267941944493-20982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 09:36:50.427311 18715 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 09:36:50.427670 18715 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 09:36:50.427758 18715 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 09:36:50.431880 18715 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 09:36:50.431973 18715 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 09:36:50.432013 18715 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 09:36:50.432070 18715 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 09:36:50.432109 18715 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 09:36:50.437978 18715 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37297
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786267941944493-20982-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786267941944493-20982-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37015
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786267941944493-20982-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37015
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786267941944493-20982-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18715
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786267941944493-20982-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 08:38:55 UTC on ccycloud.quasar-njjleu.root.comops.site
build id 4575935
I20260809 09:36:50.439313 18715 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 09:36:50.440407 18715 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 09:36:50.449430 18722 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:36:50.449457 18720 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:36:50.449430 18725 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:36:50.449710 18715 server_base.cc:1034] running on GCE node
I20260809 09:36:50.450421 18715 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 09:36:50.451565 18715 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 09:36:50.452724 18715 hybrid_clock.cc:648] HybridClock initialized: now 1786268210452698 us; error 40 us; skew 500 ppm
Aug 09 09:36:50 dist-test-slave-7kzw krb5kdc[18700](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268210, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 09:36:50.455825 18715 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 09:36:50.456733 18715 webserver.cc:460] Webserver started at http://127.0.0.1:39411/ using document root <none> and password file <none>
I20260809 09:36:50.457325 18715 fs_manager.cc:359] Metadata directory not provided
I20260809 09:36:50.457391 18715 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 09:36:50.457666 18715 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 09:36:50.459910 18715 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786267941944493-20982-0/minicluster-data/master-0/data/instance:
uuid: "eb9807333ed049d3a105be25d4eea528"
format_stamp: "Formatted at 2026-08-09 09:36:50 on dist-test-slave-7kzw"
server_key: "5d08e77fcfb042d0935e269a3d74d5ba"
server_key_iv: "0fa11fe55cab4f17ef36d08d174148f5"
server_key_version: "encryptionkey@0"
I20260809 09:36:50.460453 18715 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786267941944493-20982-0/minicluster-data/master-0/wal/instance:
uuid: "eb9807333ed049d3a105be25d4eea528"
format_stamp: "Formatted at 2026-08-09 09:36:50 on dist-test-slave-7kzw"
server_key: "5d08e77fcfb042d0935e269a3d74d5ba"
server_key_iv: "0fa11fe55cab4f17ef36d08d174148f5"
server_key_version: "encryptionkey@0"
I20260809 09:36:50.464406 18715 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.005s
I20260809 09:36:50.467103 18740 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 09:36:50.468477 18715 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 09:36:50.468585 18715 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786267941944493-20982-0/minicluster-data/master-0/data,/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786267941944493-20982-0/minicluster-data/master-0/wal
uuid: "eb9807333ed049d3a105be25d4eea528"
format_stamp: "Formatted at 2026-08-09 09:36:50 on dist-test-slave-7kzw"
server_key: "5d08e77fcfb042d0935e269a3d74d5ba"
server_key_iv: "0fa11fe55cab4f17ef36d08d174148f5"
server_key_version: "encryptionkey@0"
I20260809 09:36:50.468760 18715 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786267941944493-20982-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786267941944493-20982-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786267941944493-20982-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 09:36:50.491935 18715 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 09:36:50.495589 18715 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 09:36:50.495838 18715 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 09:36:50.504400 18715 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37015
I20260809 09:36:50.504431 18813 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37015 every 8 connection(s)
I20260809 09:36:50.505525 18715 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786267941944493-20982-0/minicluster-data/master-0/data/info.pb
I20260809 09:36:50.507956 20982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu as pid 18715
I20260809 09:36:50.508098 20982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786267941944493-20982-0/minicluster-data/master-0/wal/instance
I20260809 09:36:50.508490 20982 external_mini_cluster.cc:1428] Setting key 7722cd55e59a68fab9740cb0175eff90
I20260809 09:36:50.508880 18816 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 09:36:50.515614 18816 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eb9807333ed049d3a105be25d4eea528: Bootstrap starting.
I20260809 09:36:50.517972 18816 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P eb9807333ed049d3a105be25d4eea528: Neither blocks nor log segments found. Creating new log.
I20260809 09:36:50.518431 18825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:36:50.510053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57801 (local address 127.0.0.1:37015)
0809 09:36:50.510632 (+ 579us) server_negotiation.cc:206] Beginning negotiation
0809 09:36:50.510649 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0809 09:36:50.510689 (+ 40us) server_negotiation.cc:407] Connection header received
0809 09:36:50.511749 (+ 1060us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:36:50.511792 (+ 43us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:36:50.512143 (+ 351us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:36:50.512512 (+ 369us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:36:50.512946 (+ 434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:36:50.513940 (+ 994us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:36:50.514944 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:36:50.515425 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:36:50.515611 (+ 186us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:36:50.516616 (+ 1005us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:36:50.516621 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:36:50.516634 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 09:36:50.517063 (+ 429us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:36:50.517124 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 09:36:50.517148 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:36:50.517299 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0809 09:36:50.517500 (+ 201us) server_negotiation.cc:299] Negotiation successful
0809 09:36:50.517646 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":354,"thread_start_us":147,"threads_started":1}
I20260809 09:36:50.518913 18816 log.cc:824] T 00000000000000000000000000000000 P eb9807333ed049d3a105be25d4eea528: Log is configured to *not* fsync() on all Append() calls
I20260809 09:36:50.521276 18816 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eb9807333ed049d3a105be25d4eea528: No bootstrap required, opened a new log
I20260809 09:36:50.521768 20982 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 09:36:50.525672 18816 raft_consensus.cc:348] T 00000000000000000000000000000000 P eb9807333ed049d3a105be25d4eea528 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb9807333ed049d3a105be25d4eea528" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37015 } }
I20260809 09:36:50.525975 18816 raft_consensus.cc:374] T 00000000000000000000000000000000 P eb9807333ed049d3a105be25d4eea528 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 09:36:50.526029 18816 raft_consensus.cc:729] T 00000000000000000000000000000000 P eb9807333ed049d3a105be25d4eea528 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb9807333ed049d3a105be25d4eea528, State: Initialized, Role: FOLLOWER
I20260809 09:36:50.526516 18816 consensus_queue.cc:260] T 00000000000000000000000000000000 P eb9807333ed049d3a105be25d4eea528 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb9807333ed049d3a105be25d4eea528" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37015 } }
I20260809 09:36:50.526710 18816 raft_consensus.cc:388] T 00000000000000000000000000000000 P eb9807333ed049d3a105be25d4eea528 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 09:36:50.526784 18816 raft_consensus.cc:482] T 00000000000000000000000000000000 P eb9807333ed049d3a105be25d4eea528 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 09:36:50.526909 18816 raft_consensus.cc:3037] T 00000000000000000000000000000000 P eb9807333ed049d3a105be25d4eea528 [term 0 FOLLOWER]: Advancing to term 1
I20260809 09:36:50.527901 18816 raft_consensus.cc:504] T 00000000000000000000000000000000 P eb9807333ed049d3a105be25d4eea528 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb9807333ed049d3a105be25d4eea528" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37015 } }
I20260809 09:36:50.528272 18816 leader_election.cc:302] T 00000000000000000000000000000000 P eb9807333ed049d3a105be25d4eea528 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: eb9807333ed049d3a105be25d4eea528; no voters:
I20260809 09:36:50.528622 18825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:36:50.523971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34711 (local address 127.0.0.1:37015)
0809 09:36:50.524093 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0809 09:36:50.524097 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 09:36:50.524362 (+ 265us) server_negotiation.cc:407] Connection header received
0809 09:36:50.524448 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:36:50.524451 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:36:50.524494 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:36:50.524575 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:36:50.524894 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:36:50.525431 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:36:50.526563 (+ 1132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:36:50.526982 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:36:50.527117 (+ 135us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:36:50.528064 (+ 947us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:36:50.528068 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:36:50.528071 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 09:36:50.528084 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:36:50.528119 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 09:36:50.528130 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:36:50.528243 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0809 09:36:50.528386 (+ 143us) server_negotiation.cc:299] Negotiation successful
0809 09:36:50.528424 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260809 09:36:50.528638 18816 leader_election.cc:288] T 00000000000000000000000000000000 P eb9807333ed049d3a105be25d4eea528 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 09:36:50.528769 18833 raft_consensus.cc:2781] T 00000000000000000000000000000000 P eb9807333ed049d3a105be25d4eea528 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 09:36:50.529019 18833 raft_consensus.cc:686] T 00000000000000000000000000000000 P eb9807333ed049d3a105be25d4eea528 [term 1 LEADER]: Becoming Leader. State: Replica: eb9807333ed049d3a105be25d4eea528, State: Running, Role: LEADER
I20260809 09:36:50.529867 18833 consensus_queue.cc:237] T 00000000000000000000000000000000 P eb9807333ed049d3a105be25d4eea528 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb9807333ed049d3a105be25d4eea528" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37015 } }
I20260809 09:36:50.530226 18816 sys_catalog.cc:564] T 00000000000000000000000000000000 P eb9807333ed049d3a105be25d4eea528 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 09:36:50.531205 18836 sys_catalog.cc:455] T 00000000000000000000000000000000 P eb9807333ed049d3a105be25d4eea528 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "eb9807333ed049d3a105be25d4eea528" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb9807333ed049d3a105be25d4eea528" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37015 } } }
I20260809 09:36:50.531347 18836 sys_catalog.cc:458] T 00000000000000000000000000000000 P eb9807333ed049d3a105be25d4eea528 [sys.catalog]: This master's current role is: LEADER
I20260809 09:36:50.531632 18834 sys_catalog.cc:455] T 00000000000000000000000000000000 P eb9807333ed049d3a105be25d4eea528 [sys.catalog]: SysCatalogTable state changed. Reason: New leader eb9807333ed049d3a105be25d4eea528. Latest consensus state: current_term: 1 leader_uuid: "eb9807333ed049d3a105be25d4eea528" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb9807333ed049d3a105be25d4eea528" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37015 } } }
I20260809 09:36:50.531723 18852 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 09:36:50.531731 18834 sys_catalog.cc:458] T 00000000000000000000000000000000 P eb9807333ed049d3a105be25d4eea528 [sys.catalog]: This master's current role is: LEADER
I20260809 09:36:50.535187 18852 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 09:36:50.541826 18852 catalog_manager.cc:1347] Generated new cluster ID: 218a44c688f74ae393c3e9fc02811341
I20260809 09:36:50.541908 18852 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 09:36:50.557617 18852 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 09:36:50.558794 18852 catalog_manager.cc:1504] Loading token signing keys...
I20260809 09:36:50.566195 18852 catalog_manager.cc:5868] T 00000000000000000000000000000000 P eb9807333ed049d3a105be25d4eea528: Generated new TSK 0
I20260809 09:36:50.566964 18852 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 09:36:50.575503 20982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu with pid 18715
W20260809 09:36:50.585503 18698 connection.cc:535] client connection to 127.0.0.1:37015 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:37015): Connection reset by peer (error 104)
2026-08-09T09:36:50Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1656 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786267941944493-20982-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 09:36:50 dist-test-slave-7kzw krb5kdc[18873](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 09:36:50 dist-test-slave-7kzw krb5kdc[18873](info): set up 2 sockets
Aug 09 09:36:50 dist-test-slave-7kzw krb5kdc[18873](info): commencing operation
krb5kdc: starting...
W20260809 09:36:51.830925 20982 mini_kdc.cc:121] Time spent starting KDC: real 1.231s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786267941944493-20982-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786267941944493-20982-0/krb5kdc/test-user.keytab.
Aug 09 09:36:51 dist-test-slave-7kzw krb5kdc[18873](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268211, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T09:36:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T09:36:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786267941944493-20982-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786267941944493-20982-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 09:36:51.966377 20982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
/tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786267941944493-20982-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786267941944493-20982-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786267941944493-20982-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786267941944493-20982-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44481
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786267941944493-20982-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45157
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44481
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786267941944493-20982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786267941944493-20982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786267941944493-20982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 09:36:52.085451 18894 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 09:36:52.085780 18894 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 09:36:52.085867 18894 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 09:36:52.089972 18894 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 09:36:52.090063 18894 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 09:36:52.090102 18894 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 09:36:52.090134 18894 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 09:36:52.090166 18894 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 09:36:52.095451 18894 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45157
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786267941944493-20982-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786267941944493-20982-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44481
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786267941944493-20982-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44481
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786267941944493-20982-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18894
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786267941944493-20982-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 08:38:55 UTC on ccycloud.quasar-njjleu.root.comops.site
build id 4575935
I20260809 09:36:52.096508 18894 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 09:36:52.097527 18894 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 09:36:52.106328 18902 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:36:52.106302 18901 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:36:52.106500 18894 server_base.cc:1034] running on GCE node
W20260809 09:36:52.106685 18907 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:36:52.107240 18894 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 09:36:52.108305 18894 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 09:36:52.109481 18894 hybrid_clock.cc:648] HybridClock initialized: now 1786268212109450 us; error 44 us; skew 500 ppm
Aug 09 09:36:52 dist-test-slave-7kzw krb5kdc[18873](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268212, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 09:36:52.117463 18894 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 09:36:52.118398 18894 webserver.cc:460] Webserver started at http://127.0.0.1:45797/ using document root <none> and password file <none>
I20260809 09:36:52.119009 18894 fs_manager.cc:359] Metadata directory not provided
I20260809 09:36:52.119073 18894 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 09:36:52.119304 18894 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 09:36:52.121403 18894 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786267941944493-20982-0/minicluster-data/master-0/data/instance:
uuid: "dad11c56da4f4474b170c47eccfe9b52"
format_stamp: "Formatted at 2026-08-09 09:36:52 on dist-test-slave-7kzw"
server_key: "79c7351e3c4f9318bdc4f02018f25f51"
server_key_iv: "554b4dbb5bec2bc4c3d269d783606a03"
server_key_version: "encryptionkey@0"
I20260809 09:36:52.121989 18894 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786267941944493-20982-0/minicluster-data/master-0/wal/instance:
uuid: "dad11c56da4f4474b170c47eccfe9b52"
format_stamp: "Formatted at 2026-08-09 09:36:52 on dist-test-slave-7kzw"
server_key: "79c7351e3c4f9318bdc4f02018f25f51"
server_key_iv: "554b4dbb5bec2bc4c3d269d783606a03"
server_key_version: "encryptionkey@0"
I20260809 09:36:52.125856 18894 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260809 09:36:52.128476 18916 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 09:36:52.129776 18894 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260809 09:36:52.129923 18894 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786267941944493-20982-0/minicluster-data/master-0/data,/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786267941944493-20982-0/minicluster-data/master-0/wal
uuid: "dad11c56da4f4474b170c47eccfe9b52"
format_stamp: "Formatted at 2026-08-09 09:36:52 on dist-test-slave-7kzw"
server_key: "79c7351e3c4f9318bdc4f02018f25f51"
server_key_iv: "554b4dbb5bec2bc4c3d269d783606a03"
server_key_version: "encryptionkey@0"
I20260809 09:36:52.130039 18894 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786267941944493-20982-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786267941944493-20982-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786267941944493-20982-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 09:36:52.135114 18894 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 09:36:52.138125 18894 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 09:36:52.138332 18894 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 09:36:52.145565 18894 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44481
I20260809 09:36:52.145574 19003 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44481 every 8 connection(s)
I20260809 09:36:52.146574 18894 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786267941944493-20982-0/minicluster-data/master-0/data/info.pb
I20260809 09:36:52.149360 19004 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 09:36:52.153609 20982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu as pid 18894
I20260809 09:36:52.153797 20982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786267941944493-20982-0/minicluster-data/master-0/wal/instance
I20260809 09:36:52.154102 20982 external_mini_cluster.cc:1428] Setting key 53ed1f341665b93297eeda0a32d8757b
I20260809 09:36:52.155320 19004 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dad11c56da4f4474b170c47eccfe9b52: Bootstrap starting.
I20260809 09:36:52.157689 19004 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dad11c56da4f4474b170c47eccfe9b52: Neither blocks nor log segments found. Creating new log.
I20260809 09:36:52.158643 19004 log.cc:824] T 00000000000000000000000000000000 P dad11c56da4f4474b170c47eccfe9b52: Log is configured to *not* fsync() on all Append() calls
I20260809 09:36:52.159370 19007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:36:52.155467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58601 (local address 127.0.0.1:44481)
0809 09:36:52.155880 (+ 413us) server_negotiation.cc:206] Beginning negotiation
0809 09:36:52.155895 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 09:36:52.155931 (+ 36us) server_negotiation.cc:407] Connection header received
0809 09:36:52.156540 (+ 609us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:36:52.156556 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:36:52.156825 (+ 269us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:36:52.157094 (+ 269us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:36:52.157942 (+ 848us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:36:52.157946 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:36:52.157957 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 09:36:52.157968 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:36:52.158021 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 09:36:52.158039 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:36:52.158127 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0809 09:36:52.158256 (+ 129us) server_negotiation.cc:299] Negotiation successful
0809 09:36:52.158418 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":251,"thread_start_us":120,"threads_started":1}
I20260809 09:36:52.160759 19004 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dad11c56da4f4474b170c47eccfe9b52: No bootstrap required, opened a new log
I20260809 09:36:52.162297 20982 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 09:36:52.165485 19004 raft_consensus.cc:348] T 00000000000000000000000000000000 P dad11c56da4f4474b170c47eccfe9b52 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dad11c56da4f4474b170c47eccfe9b52" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44481 } }
I20260809 09:36:52.165766 19004 raft_consensus.cc:374] T 00000000000000000000000000000000 P dad11c56da4f4474b170c47eccfe9b52 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 09:36:52.165831 19004 raft_consensus.cc:729] T 00000000000000000000000000000000 P dad11c56da4f4474b170c47eccfe9b52 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dad11c56da4f4474b170c47eccfe9b52, State: Initialized, Role: FOLLOWER
I20260809 09:36:52.166373 19007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:36:52.164692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50075 (local address 127.0.0.1:44481)
0809 09:36:52.164810 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0809 09:36:52.164814 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 09:36:52.165007 (+ 193us) server_negotiation.cc:407] Connection header received
0809 09:36:52.165117 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:36:52.165119 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:36:52.165146 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:36:52.165221 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:36:52.165953 (+ 732us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:36:52.165956 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:36:52.165959 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 09:36:52.165970 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:36:52.166001 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 09:36:52.166011 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:36:52.166112 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0809 09:36:52.166213 (+ 101us) server_negotiation.cc:299] Negotiation successful
0809 09:36:52.166244 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260809 09:36:52.166329 19004 consensus_queue.cc:260] T 00000000000000000000000000000000 P dad11c56da4f4474b170c47eccfe9b52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dad11c56da4f4474b170c47eccfe9b52" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44481 } }
I20260809 09:36:52.166522 19004 raft_consensus.cc:388] T 00000000000000000000000000000000 P dad11c56da4f4474b170c47eccfe9b52 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 09:36:52.166596 19004 raft_consensus.cc:482] T 00000000000000000000000000000000 P dad11c56da4f4474b170c47eccfe9b52 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 09:36:52.166690 19004 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dad11c56da4f4474b170c47eccfe9b52 [term 0 FOLLOWER]: Advancing to term 1
I20260809 09:36:52.167830 19004 raft_consensus.cc:504] T 00000000000000000000000000000000 P dad11c56da4f4474b170c47eccfe9b52 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dad11c56da4f4474b170c47eccfe9b52" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44481 } }
I20260809 09:36:52.168196 19004 leader_election.cc:302] T 00000000000000000000000000000000 P dad11c56da4f4474b170c47eccfe9b52 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dad11c56da4f4474b170c47eccfe9b52; no voters:
I20260809 09:36:52.168529 19004 leader_election.cc:288] T 00000000000000000000000000000000 P dad11c56da4f4474b170c47eccfe9b52 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 09:36:52.168594 19016 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dad11c56da4f4474b170c47eccfe9b52 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 09:36:52.168825 19016 raft_consensus.cc:686] T 00000000000000000000000000000000 P dad11c56da4f4474b170c47eccfe9b52 [term 1 LEADER]: Becoming Leader. State: Replica: dad11c56da4f4474b170c47eccfe9b52, State: Running, Role: LEADER
I20260809 09:36:52.169128 19016 consensus_queue.cc:237] T 00000000000000000000000000000000 P dad11c56da4f4474b170c47eccfe9b52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dad11c56da4f4474b170c47eccfe9b52" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44481 } }
I20260809 09:36:52.169965 19004 sys_catalog.cc:564] T 00000000000000000000000000000000 P dad11c56da4f4474b170c47eccfe9b52 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 09:36:52.171985 19017 sys_catalog.cc:455] T 00000000000000000000000000000000 P dad11c56da4f4474b170c47eccfe9b52 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dad11c56da4f4474b170c47eccfe9b52. Latest consensus state: current_term: 1 leader_uuid: "dad11c56da4f4474b170c47eccfe9b52" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dad11c56da4f4474b170c47eccfe9b52" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44481 } } }
I20260809 09:36:52.172113 19017 sys_catalog.cc:458] T 00000000000000000000000000000000 P dad11c56da4f4474b170c47eccfe9b52 [sys.catalog]: This master's current role is: LEADER
I20260809 09:36:52.171967 19018 sys_catalog.cc:455] T 00000000000000000000000000000000 P dad11c56da4f4474b170c47eccfe9b52 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dad11c56da4f4474b170c47eccfe9b52" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dad11c56da4f4474b170c47eccfe9b52" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44481 } } }
I20260809 09:36:52.172423 19018 sys_catalog.cc:458] T 00000000000000000000000000000000 P dad11c56da4f4474b170c47eccfe9b52 [sys.catalog]: This master's current role is: LEADER
W20260809 09:36:52.175033 19038 catalog_manager.cc:1558] T 00000000000000000000000000000000 P dad11c56da4f4474b170c47eccfe9b52: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 09:36:52.175124 19038 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 09:36:52.175199 19037 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 09:36:52.175870 19037 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 09:36:52.181330 19037 catalog_manager.cc:1347] Generated new cluster ID: 65b95333e08f4dc1b1a6cbeb08e28c7a
I20260809 09:36:52.181407 19037 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 09:36:52.191599 19037 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 09:36:52.191939 19037 catalog_manager.cc:1504] Loading token signing keys...
I20260809 09:36:52.200421 19037 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dad11c56da4f4474b170c47eccfe9b52: Generated new TSK 0
I20260809 09:36:52.201112 19037 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 09:36:52.209224 20982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu with pid 18894
2026-08-09T09:36:52Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1637 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786267941944493-20982-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 09:36:52 dist-test-slave-7kzw krb5kdc[19048](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 09:36:52 dist-test-slave-7kzw krb5kdc[19048](info): set up 2 sockets
Aug 09 09:36:52 dist-test-slave-7kzw krb5kdc[19048](info): commencing operation
krb5kdc: starting...
W20260809 09:36:53.442023 20982 mini_kdc.cc:121] Time spent starting KDC: real 1.204s user 0.001s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786267941944493-20982-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786267941944493-20982-0/krb5kdc/test-user.keytab.
Aug 09 09:36:53 dist-test-slave-7kzw krb5kdc[19048](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268213, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T09:36:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T09:36:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786267941944493-20982-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786267941944493-20982-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 09:36:53.578411 20982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
/tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786267941944493-20982-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786267941944493-20982-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786267941944493-20982-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786267941944493-20982-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46529
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786267941944493-20982-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39245
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46529
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786267941944493-20982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786267941944493-20982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786267941944493-20982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 09:36:53.698242 19067 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 09:36:53.698575 19067 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 09:36:53.698657 19067 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 09:36:53.702611 19067 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 09:36:53.702677 19067 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 09:36:53.702697 19067 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 09:36:53.702714 19067 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 09:36:53.702729 19067 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 09:36:53.708246 19067 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39245
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786267941944493-20982-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786267941944493-20982-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46529
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786267941944493-20982-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46529
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786267941944493-20982-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19067
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786267941944493-20982-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 08:38:55 UTC on ccycloud.quasar-njjleu.root.comops.site
build id 4575935
I20260809 09:36:53.709396 19067 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 09:36:53.710489 19067 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 09:36:53.719385 19076 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:36:53.719352 19074 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:36:53.719518 19067 server_base.cc:1034] running on GCE node
W20260809 09:36:53.719746 19081 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:36:53.720388 19067 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 09:36:53.721570 19067 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 09:36:53.722774 19067 hybrid_clock.cc:648] HybridClock initialized: now 1786268213722738 us; error 46 us; skew 500 ppm
Aug 09 09:36:53 dist-test-slave-7kzw krb5kdc[19048](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268213, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 09:36:53.725725 19067 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 09:36:53.726713 19067 webserver.cc:460] Webserver started at http://127.0.0.1:42481/ using document root <none> and password file <none>
I20260809 09:36:53.727304 19067 fs_manager.cc:359] Metadata directory not provided
I20260809 09:36:53.727402 19067 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 09:36:53.727645 19067 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 09:36:53.729718 19067 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786267941944493-20982-0/minicluster-data/master-0/data/instance:
uuid: "e0ff1de7037a41d4ab974fe4df640144"
format_stamp: "Formatted at 2026-08-09 09:36:53 on dist-test-slave-7kzw"
server_key: "0bde644cde639ac83d403a3e45cb515a"
server_key_iv: "5f68db216af17193a90b400b009bdf9d"
server_key_version: "encryptionkey@0"
I20260809 09:36:53.730549 19067 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786267941944493-20982-0/minicluster-data/master-0/wal/instance:
uuid: "e0ff1de7037a41d4ab974fe4df640144"
format_stamp: "Formatted at 2026-08-09 09:36:53 on dist-test-slave-7kzw"
server_key: "0bde644cde639ac83d403a3e45cb515a"
server_key_iv: "5f68db216af17193a90b400b009bdf9d"
server_key_version: "encryptionkey@0"
I20260809 09:36:53.734524 19067 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260809 09:36:53.737069 19090 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 09:36:53.738529 19067 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 09:36:53.738700 19067 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786267941944493-20982-0/minicluster-data/master-0/data,/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786267941944493-20982-0/minicluster-data/master-0/wal
uuid: "e0ff1de7037a41d4ab974fe4df640144"
format_stamp: "Formatted at 2026-08-09 09:36:53 on dist-test-slave-7kzw"
server_key: "0bde644cde639ac83d403a3e45cb515a"
server_key_iv: "5f68db216af17193a90b400b009bdf9d"
server_key_version: "encryptionkey@0"
I20260809 09:36:53.738821 19067 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786267941944493-20982-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786267941944493-20982-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786267941944493-20982-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 09:36:53.758078 19067 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 09:36:53.761261 19067 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 09:36:53.761484 19067 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 09:36:53.768795 19067 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46529
I20260809 09:36:53.768959 19161 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46529 every 8 connection(s)
I20260809 09:36:53.769814 19067 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786267941944493-20982-0/minicluster-data/master-0/data/info.pb
I20260809 09:36:53.772624 19162 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 09:36:53.776713 20982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu as pid 19067
I20260809 09:36:53.776872 20982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786267941944493-20982-0/minicluster-data/master-0/wal/instance
I20260809 09:36:53.777218 20982 external_mini_cluster.cc:1428] Setting key 21f44e66f449b0e2176a10146fe17b70
I20260809 09:36:53.778957 19162 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e0ff1de7037a41d4ab974fe4df640144: Bootstrap starting.
I20260809 09:36:53.781213 19162 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e0ff1de7037a41d4ab974fe4df640144: Neither blocks nor log segments found. Creating new log.
I20260809 09:36:53.781978 19162 log.cc:824] T 00000000000000000000000000000000 P e0ff1de7037a41d4ab974fe4df640144: Log is configured to *not* fsync() on all Append() calls
I20260809 09:36:53.783887 19162 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e0ff1de7037a41d4ab974fe4df640144: No bootstrap required, opened a new log
Aug 09 09:36:53 dist-test-slave-7kzw krb5kdc[19048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268213, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260809 09:36:53.788102 19162 raft_consensus.cc:348] T 00000000000000000000000000000000 P e0ff1de7037a41d4ab974fe4df640144 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0ff1de7037a41d4ab974fe4df640144" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46529 } }
I20260809 09:36:53.788390 19162 raft_consensus.cc:374] T 00000000000000000000000000000000 P e0ff1de7037a41d4ab974fe4df640144 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 09:36:53.788468 19162 raft_consensus.cc:729] T 00000000000000000000000000000000 P e0ff1de7037a41d4ab974fe4df640144 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e0ff1de7037a41d4ab974fe4df640144, State: Initialized, Role: FOLLOWER
I20260809 09:36:53.788981 19162 consensus_queue.cc:260] T 00000000000000000000000000000000 P e0ff1de7037a41d4ab974fe4df640144 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0ff1de7037a41d4ab974fe4df640144" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46529 } }
I20260809 09:36:53.789109 19162 raft_consensus.cc:388] T 00000000000000000000000000000000 P e0ff1de7037a41d4ab974fe4df640144 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 09:36:53.789196 19162 raft_consensus.cc:482] T 00000000000000000000000000000000 P e0ff1de7037a41d4ab974fe4df640144 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 09:36:53.789327 19162 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e0ff1de7037a41d4ab974fe4df640144 [term 0 FOLLOWER]: Advancing to term 1
I20260809 09:36:53.790381 19162 raft_consensus.cc:504] T 00000000000000000000000000000000 P e0ff1de7037a41d4ab974fe4df640144 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0ff1de7037a41d4ab974fe4df640144" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46529 } }
I20260809 09:36:53.790763 19162 leader_election.cc:302] T 00000000000000000000000000000000 P e0ff1de7037a41d4ab974fe4df640144 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e0ff1de7037a41d4ab974fe4df640144; no voters:
I20260809 09:36:53.791121 19162 leader_election.cc:288] T 00000000000000000000000000000000 P e0ff1de7037a41d4ab974fe4df640144 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 09:36:53.791193 19167 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e0ff1de7037a41d4ab974fe4df640144 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 09:36:53.791417 19167 raft_consensus.cc:686] T 00000000000000000000000000000000 P e0ff1de7037a41d4ab974fe4df640144 [term 1 LEADER]: Becoming Leader. State: Replica: e0ff1de7037a41d4ab974fe4df640144, State: Running, Role: LEADER
I20260809 09:36:53.791754 19167 consensus_queue.cc:237] T 00000000000000000000000000000000 P e0ff1de7037a41d4ab974fe4df640144 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0ff1de7037a41d4ab974fe4df640144" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46529 } }
I20260809 09:36:53.792411 19162 sys_catalog.cc:564] T 00000000000000000000000000000000 P e0ff1de7037a41d4ab974fe4df640144 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 09:36:53.793025 19168 sys_catalog.cc:455] T 00000000000000000000000000000000 P e0ff1de7037a41d4ab974fe4df640144 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e0ff1de7037a41d4ab974fe4df640144. Latest consensus state: current_term: 1 leader_uuid: "e0ff1de7037a41d4ab974fe4df640144" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0ff1de7037a41d4ab974fe4df640144" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46529 } } }
I20260809 09:36:53.793062 19169 sys_catalog.cc:455] T 00000000000000000000000000000000 P e0ff1de7037a41d4ab974fe4df640144 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e0ff1de7037a41d4ab974fe4df640144" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0ff1de7037a41d4ab974fe4df640144" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46529 } } }
I20260809 09:36:53.793076 19165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:36:53.778617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54395 (local address 127.0.0.1:46529)
0809 09:36:53.779191 (+ 574us) server_negotiation.cc:206] Beginning negotiation
0809 09:36:53.779215 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0809 09:36:53.779258 (+ 43us) server_negotiation.cc:407] Connection header received
0809 09:36:53.779983 (+ 725us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:36:53.780001 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:36:53.780371 (+ 370us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:36:53.780758 (+ 387us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:36:53.781825 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:36:53.783066 (+ 1241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:36:53.784091 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:36:53.784765 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:36:53.784938 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:36:53.787747 (+ 2809us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:36:53.787753 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:36:53.787769 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:36:53.787799 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:36:53.789807 (+ 2008us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:36:53.790333 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:36:53.790338 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:36:53.790343 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:36:53.790476 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:36:53.790812 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:36:53.790816 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:36:53.790818 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:36:53.791288 (+ 470us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:36:53.791481 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0809 09:36:53.791790 (+ 309us) server_negotiation.cc:299] Negotiation successful
0809 09:36:53.792116 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":342,"thread_start_us":139,"threads_started":1}
I20260809 09:36:53.793200 19168 sys_catalog.cc:458] T 00000000000000000000000000000000 P e0ff1de7037a41d4ab974fe4df640144 [sys.catalog]: This master's current role is: LEADER
I20260809 09:36:53.793208 19169 sys_catalog.cc:458] T 00000000000000000000000000000000 P e0ff1de7037a41d4ab974fe4df640144 [sys.catalog]: This master's current role is: LEADER
I20260809 09:36:53.796545 20982 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260809 09:36:53.797488 19188 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e0ff1de7037a41d4ab974fe4df640144: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 09:36:53.797571 19188 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 09:36:53.797652 19174 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 09:36:53.798619 19174 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 09:36:53.809473 19174 catalog_manager.cc:1347] Generated new cluster ID: 4a9a79e1ce5e4f04bf803dafa22c6337
I20260809 09:36:53.810150 19174 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 09:36:53.815838 19165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:36:53.798646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53781 (local address 127.0.0.1:46529)
0809 09:36:53.798749 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0809 09:36:53.798754 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 09:36:53.800258 (+ 1504us) server_negotiation.cc:407] Connection header received
0809 09:36:53.800465 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:36:53.800471 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:36:53.800531 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:36:53.800592 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:36:53.801703 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:36:53.802524 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:36:53.804254 (+ 1730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:36:53.804620 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:36:53.804735 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:36:53.807617 (+ 2882us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:36:53.807622 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:36:53.807625 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:36:53.807652 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:36:53.809503 (+ 1851us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:36:53.810790 (+ 1287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:36:53.810794 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:36:53.810796 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:36:53.810891 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:36:53.814816 (+ 3925us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:36:53.814820 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:36:53.814822 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:36:53.815060 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:36:53.815211 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0809 09:36:53.815509 (+ 298us) server_negotiation.cc:299] Negotiation successful
0809 09:36:53.815645 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260809 09:36:53.816965 19174 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 09:36:53.817831 19174 catalog_manager.cc:1504] Loading token signing keys...
I20260809 09:36:53.823886 19174 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e0ff1de7037a41d4ab974fe4df640144: Generated new TSK 0
I20260809 09:36:53.824520 19174 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 09:36:53.832914 20982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu with pid 19067
2026-08-09T09:36:53Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1626 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786267941944493-20982-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 09:36:53 dist-test-slave-7kzw krb5kdc[19202](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 09:36:53 dist-test-slave-7kzw krb5kdc[19202](info): set up 2 sockets
Aug 09 09:36:53 dist-test-slave-7kzw krb5kdc[19202](info): commencing operation
krb5kdc: starting...
W20260809 09:36:55.075800 20982 mini_kdc.cc:121] Time spent starting KDC: real 1.211s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786267941944493-20982-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786267941944493-20982-0/krb5kdc/test-user.keytab.
Aug 09 09:36:55 dist-test-slave-7kzw krb5kdc[19202](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268215, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T09:36:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T09:36:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786267941944493-20982-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786267941944493-20982-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 09:36:55.218012 20982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
/tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786267941944493-20982-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786267941944493-20982-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786267941944493-20982-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786267941944493-20982-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37709
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786267941944493-20982-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35275
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37709
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786267941944493-20982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786267941944493-20982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786267941944493-20982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 09:36:55.333230 19221 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 09:36:55.333508 19221 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 09:36:55.333550 19221 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 09:36:55.337554 19221 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 09:36:55.337620 19221 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 09:36:55.337641 19221 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 09:36:55.337656 19221 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 09:36:55.337671 19221 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 09:36:55.342829 19221 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35275
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786267941944493-20982-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786267941944493-20982-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37709
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786267941944493-20982-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37709
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786267941944493-20982-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19221
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786267941944493-20982-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 08:38:55 UTC on ccycloud.quasar-njjleu.root.comops.site
build id 4575935
I20260809 09:36:55.343797 19221 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 09:36:55.344759 19221 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 09:36:55.353634 19233 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:36:55.353605 19237 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:36:55.353610 19232 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:36:55.353904 19221 server_base.cc:1034] running on GCE node
I20260809 09:36:55.354653 19221 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 09:36:55.355805 19221 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 09:36:55.356971 19221 hybrid_clock.cc:648] HybridClock initialized: now 1786268215356950 us; error 53 us; skew 500 ppm
Aug 09 09:36:55 dist-test-slave-7kzw krb5kdc[19202](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268215, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 09:36:55.359930 19221 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 09:36:55.360827 19221 webserver.cc:460] Webserver started at http://127.0.0.1:44675/ using document root <none> and password file <none>
I20260809 09:36:55.361414 19221 fs_manager.cc:359] Metadata directory not provided
I20260809 09:36:55.361482 19221 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 09:36:55.361743 19221 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 09:36:55.363842 19221 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786267941944493-20982-0/minicluster-data/master-0/data/instance:
uuid: "e55d0eab95ca4383b6ac90f99236ef3b"
format_stamp: "Formatted at 2026-08-09 09:36:55 on dist-test-slave-7kzw"
server_key: "87374840258d1c44bec69681d949daf8"
server_key_iv: "5c67b8fe9080c66b510b385104117c24"
server_key_version: "encryptionkey@0"
I20260809 09:36:55.364421 19221 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786267941944493-20982-0/minicluster-data/master-0/wal/instance:
uuid: "e55d0eab95ca4383b6ac90f99236ef3b"
format_stamp: "Formatted at 2026-08-09 09:36:55 on dist-test-slave-7kzw"
server_key: "87374840258d1c44bec69681d949daf8"
server_key_iv: "5c67b8fe9080c66b510b385104117c24"
server_key_version: "encryptionkey@0"
I20260809 09:36:55.368301 19221 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 09:36:55.370872 19246 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 09:36:55.372359 19221 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 09:36:55.372536 19221 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786267941944493-20982-0/minicluster-data/master-0/data,/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786267941944493-20982-0/minicluster-data/master-0/wal
uuid: "e55d0eab95ca4383b6ac90f99236ef3b"
format_stamp: "Formatted at 2026-08-09 09:36:55 on dist-test-slave-7kzw"
server_key: "87374840258d1c44bec69681d949daf8"
server_key_iv: "5c67b8fe9080c66b510b385104117c24"
server_key_version: "encryptionkey@0"
I20260809 09:36:55.372673 19221 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786267941944493-20982-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786267941944493-20982-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786267941944493-20982-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 09:36:55.404265 19221 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 09:36:55.408141 19221 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 09:36:55.408418 19221 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 09:36:55.416567 19221 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37709
I20260809 09:36:55.416590 19329 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37709 every 8 connection(s)
I20260809 09:36:55.417640 19221 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786267941944493-20982-0/minicluster-data/master-0/data/info.pb
I20260809 09:36:55.420503 19331 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 09:36:55.426072 19331 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e55d0eab95ca4383b6ac90f99236ef3b: Bootstrap starting.
I20260809 09:36:55.426139 20982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu as pid 19221
I20260809 09:36:55.426338 20982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786267941944493-20982-0/minicluster-data/master-0/wal/instance
I20260809 09:36:55.426739 20982 external_mini_cluster.cc:1428] Setting key ad1d626a0fa7366e94ecbcabf363f0d2
I20260809 09:36:55.428866 19331 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e55d0eab95ca4383b6ac90f99236ef3b: Neither blocks nor log segments found. Creating new log.
I20260809 09:36:55.429765 19331 log.cc:824] T 00000000000000000000000000000000 P e55d0eab95ca4383b6ac90f99236ef3b: Log is configured to *not* fsync() on all Append() calls
I20260809 09:36:55.432024 19331 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e55d0eab95ca4383b6ac90f99236ef3b: No bootstrap required, opened a new log
I20260809 09:36:55.436183 19331 raft_consensus.cc:348] T 00000000000000000000000000000000 P e55d0eab95ca4383b6ac90f99236ef3b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e55d0eab95ca4383b6ac90f99236ef3b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37709 } }
I20260809 09:36:55.436462 19331 raft_consensus.cc:374] T 00000000000000000000000000000000 P e55d0eab95ca4383b6ac90f99236ef3b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 09:36:55.436538 19331 raft_consensus.cc:729] T 00000000000000000000000000000000 P e55d0eab95ca4383b6ac90f99236ef3b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e55d0eab95ca4383b6ac90f99236ef3b, State: Initialized, Role: FOLLOWER
I20260809 09:36:55.437043 19331 consensus_queue.cc:260] T 00000000000000000000000000000000 P e55d0eab95ca4383b6ac90f99236ef3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e55d0eab95ca4383b6ac90f99236ef3b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37709 } }
I20260809 09:36:55.437203 19331 raft_consensus.cc:388] T 00000000000000000000000000000000 P e55d0eab95ca4383b6ac90f99236ef3b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 09:36:55.437278 19331 raft_consensus.cc:482] T 00000000000000000000000000000000 P e55d0eab95ca4383b6ac90f99236ef3b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 09:36:55.437378 19331 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e55d0eab95ca4383b6ac90f99236ef3b [term 0 FOLLOWER]: Advancing to term 1
I20260809 09:36:55.437517 19334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:36:55.428293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57251 (local address 127.0.0.1:37709)
0809 09:36:55.428750 (+ 457us) server_negotiation.cc:206] Beginning negotiation
0809 09:36:55.428772 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0809 09:36:55.428821 (+ 49us) server_negotiation.cc:407] Connection header received
0809 09:36:55.429703 (+ 882us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:36:55.429727 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:36:55.430170 (+ 443us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:36:55.430530 (+ 360us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:36:55.430994 (+ 464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:36:55.432437 (+ 1443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:36:55.433475 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:36:55.434190 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:36:55.434354 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:36:55.435362 (+ 1008us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:36:55.435371 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:36:55.435391 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 09:36:55.435890 (+ 499us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:36:55.435957 (+ 67us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 09:36:55.435988 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:36:55.436088 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0809 09:36:55.436278 (+ 190us) server_negotiation.cc:299] Negotiation successful
0809 09:36:55.436489 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":291,"thread_start_us":119,"threads_started":1}
I20260809 09:36:55.438359 19331 raft_consensus.cc:504] T 00000000000000000000000000000000 P e55d0eab95ca4383b6ac90f99236ef3b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e55d0eab95ca4383b6ac90f99236ef3b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37709 } }
I20260809 09:36:55.438797 19331 leader_election.cc:302] T 00000000000000000000000000000000 P e55d0eab95ca4383b6ac90f99236ef3b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e55d0eab95ca4383b6ac90f99236ef3b; no voters:
I20260809 09:36:55.439133 19331 leader_election.cc:288] T 00000000000000000000000000000000 P e55d0eab95ca4383b6ac90f99236ef3b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 09:36:55.439234 19336 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e55d0eab95ca4383b6ac90f99236ef3b [term 1 FOLLOWER]: Leader election won for term 1
I20260809 09:36:55.439530 19336 raft_consensus.cc:686] T 00000000000000000000000000000000 P e55d0eab95ca4383b6ac90f99236ef3b [term 1 LEADER]: Becoming Leader. State: Replica: e55d0eab95ca4383b6ac90f99236ef3b, State: Running, Role: LEADER
I20260809 09:36:55.439952 19336 consensus_queue.cc:237] T 00000000000000000000000000000000 P e55d0eab95ca4383b6ac90f99236ef3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e55d0eab95ca4383b6ac90f99236ef3b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37709 } }
I20260809 09:36:55.440476 19331 sys_catalog.cc:564] T 00000000000000000000000000000000 P e55d0eab95ca4383b6ac90f99236ef3b [sys.catalog]: configured and running, proceeding with master startup.
I20260809 09:36:55.440806 20982 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 09:36:55.441890 19340 sys_catalog.cc:455] T 00000000000000000000000000000000 P e55d0eab95ca4383b6ac90f99236ef3b [sys.catalog]: SysCatalogTable state changed. Reason: New leader e55d0eab95ca4383b6ac90f99236ef3b. Latest consensus state: current_term: 1 leader_uuid: "e55d0eab95ca4383b6ac90f99236ef3b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e55d0eab95ca4383b6ac90f99236ef3b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37709 } } }
I20260809 09:36:55.441896 19339 sys_catalog.cc:455] T 00000000000000000000000000000000 P e55d0eab95ca4383b6ac90f99236ef3b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e55d0eab95ca4383b6ac90f99236ef3b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e55d0eab95ca4383b6ac90f99236ef3b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37709 } } }
I20260809 09:36:55.442039 19339 sys_catalog.cc:458] T 00000000000000000000000000000000 P e55d0eab95ca4383b6ac90f99236ef3b [sys.catalog]: This master's current role is: LEADER
I20260809 09:36:55.442039 19340 sys_catalog.cc:458] T 00000000000000000000000000000000 P e55d0eab95ca4383b6ac90f99236ef3b [sys.catalog]: This master's current role is: LEADER
W20260809 09:36:55.445703 19365 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e55d0eab95ca4383b6ac90f99236ef3b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 09:36:55.445838 19365 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 09:36:55.445941 19355 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 09:36:55.446907 19355 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 09:36:55.448283 19334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:36:55.443680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40939 (local address 127.0.0.1:37709)
0809 09:36:55.443791 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0809 09:36:55.443795 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 09:36:55.444030 (+ 235us) server_negotiation.cc:407] Connection header received
0809 09:36:55.444239 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:36:55.444243 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:36:55.444301 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:36:55.444352 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:36:55.444778 (+ 426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:36:55.445553 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:36:55.446599 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:36:55.446838 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:36:55.446961 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:36:55.447833 (+ 872us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:36:55.447837 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:36:55.447840 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 09:36:55.447853 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:36:55.447885 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 09:36:55.447897 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:36:55.447980 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0809 09:36:55.448073 (+ 93us) server_negotiation.cc:299] Negotiation successful
0809 09:36:55.448123 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260809 09:36:55.452706 19355 catalog_manager.cc:1347] Generated new cluster ID: 5226f71518794aa9b792ba3d4208e7d5
I20260809 09:36:55.452797 19355 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 09:36:55.467569 19355 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 09:36:55.468585 19355 catalog_manager.cc:1504] Loading token signing keys...
I20260809 09:36:55.474713 19355 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e55d0eab95ca4383b6ac90f99236ef3b: Generated new TSK 0
I20260809 09:36:55.475514 19355 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 09:36:55.481570 20982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu with pid 19221
2026-08-09T09:36:55Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1640 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786267941944493-20982-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 09:36:55 dist-test-slave-7kzw krb5kdc[19375](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 09:36:55 dist-test-slave-7kzw krb5kdc[19375](info): set up 2 sockets
Aug 09 09:36:55 dist-test-slave-7kzw krb5kdc[19375](info): commencing operation
krb5kdc: starting...
W20260809 09:36:56.794452 20982 mini_kdc.cc:121] Time spent starting KDC: real 1.289s user 0.001s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786267941944493-20982-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786267941944493-20982-0/krb5kdc/test-user.keytab.
Aug 09 09:36:56 dist-test-slave-7kzw krb5kdc[19375](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268216, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T09:36:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T09:36:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786267941944493-20982-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786267941944493-20982-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 09:36:56.930883 20982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
/tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786267941944493-20982-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786267941944493-20982-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786267941944493-20982-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786267941944493-20982-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39121
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786267941944493-20982-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33955
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39121
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786267941944493-20982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786267941944493-20982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786267941944493-20982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 09:36:57.046079 19393 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 09:36:57.046447 19393 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 09:36:57.046561 19393 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 09:36:57.050705 19393 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 09:36:57.050822 19393 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 09:36:57.050861 19393 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 09:36:57.050900 19393 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 09:36:57.050932 19393 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 09:36:57.056345 19393 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33955
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786267941944493-20982-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786267941944493-20982-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39121
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786267941944493-20982-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39121
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786267941944493-20982-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19393
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786267941944493-20982-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 08:38:55 UTC on ccycloud.quasar-njjleu.root.comops.site
build id 4575935
I20260809 09:36:57.057499 19393 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 09:36:57.058619 19393 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 09:36:57.067786 19399 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:36:57.067983 19402 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:36:57.067817 19400 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:36:57.068621 19393 server_base.cc:1034] running on GCE node
I20260809 09:36:57.069298 19393 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 09:36:57.070636 19393 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 09:36:57.071869 19393 hybrid_clock.cc:648] HybridClock initialized: now 1786268217071850 us; error 37 us; skew 500 ppm
Aug 09 09:36:57 dist-test-slave-7kzw krb5kdc[19375](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268217, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 09:36:57.075237 19393 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 09:36:57.076275 19393 webserver.cc:460] Webserver started at http://127.0.0.1:42309/ using document root <none> and password file <none>
I20260809 09:36:57.076856 19393 fs_manager.cc:359] Metadata directory not provided
I20260809 09:36:57.076922 19393 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 09:36:57.077114 19393 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 09:36:57.079417 19393 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786267941944493-20982-0/minicluster-data/master-0/data/instance:
uuid: "c09c4f15c57044c59d9964440949aaf8"
format_stamp: "Formatted at 2026-08-09 09:36:57 on dist-test-slave-7kzw"
server_key: "8bd0e03beff265ccfbeb415db9c6e7b8"
server_key_iv: "5eb6077a9018d83197beb1d7ac26b907"
server_key_version: "encryptionkey@0"
I20260809 09:36:57.079981 19393 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786267941944493-20982-0/minicluster-data/master-0/wal/instance:
uuid: "c09c4f15c57044c59d9964440949aaf8"
format_stamp: "Formatted at 2026-08-09 09:36:57 on dist-test-slave-7kzw"
server_key: "8bd0e03beff265ccfbeb415db9c6e7b8"
server_key_iv: "5eb6077a9018d83197beb1d7ac26b907"
server_key_version: "encryptionkey@0"
I20260809 09:36:57.084131 19393 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 09:36:57.086913 19411 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 09:36:57.088542 19393 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 09:36:57.088662 19393 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786267941944493-20982-0/minicluster-data/master-0/data,/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786267941944493-20982-0/minicluster-data/master-0/wal
uuid: "c09c4f15c57044c59d9964440949aaf8"
format_stamp: "Formatted at 2026-08-09 09:36:57 on dist-test-slave-7kzw"
server_key: "8bd0e03beff265ccfbeb415db9c6e7b8"
server_key_iv: "5eb6077a9018d83197beb1d7ac26b907"
server_key_version: "encryptionkey@0"
I20260809 09:36:57.088769 19393 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786267941944493-20982-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786267941944493-20982-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786267941944493-20982-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 09:36:57.093461 19393 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 09:36:57.096588 19393 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 09:36:57.096740 19393 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 09:36:57.104174 19393 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39121
I20260809 09:36:57.104203 19491 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39121 every 8 connection(s)
I20260809 09:36:57.105123 19393 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786267941944493-20982-0/minicluster-data/master-0/data/info.pb
I20260809 09:36:57.108059 19492 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 09:36:57.108318 20982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu as pid 19393
I20260809 09:36:57.108465 20982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786267941944493-20982-0/minicluster-data/master-0/wal/instance
I20260809 09:36:57.108793 20982 external_mini_cluster.cc:1428] Setting key a1faca11c5d84fe6d1c16b7793eccd92
I20260809 09:36:57.114543 19495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:36:57.110257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54947 (local address 127.0.0.1:39121)
0809 09:36:57.110735 (+ 478us) server_negotiation.cc:206] Beginning negotiation
0809 09:36:57.110754 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0809 09:36:57.110798 (+ 44us) server_negotiation.cc:407] Connection header received
0809 09:36:57.111555 (+ 757us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:36:57.111581 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:36:57.111942 (+ 361us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:36:57.112332 (+ 390us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:36:57.113113 (+ 781us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:36:57.113119 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:36:57.113133 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 09:36:57.113146 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:36:57.113219 (+ 73us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 09:36:57.113248 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:36:57.113392 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0809 09:36:57.113461 (+ 69us) server_negotiation.cc:299] Negotiation successful
0809 09:36:57.113611 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":322,"thread_start_us":178,"threads_started":1}
I20260809 09:36:57.115494 19492 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c09c4f15c57044c59d9964440949aaf8: Bootstrap starting.
I20260809 09:36:57.117395 20982 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 09:36:57.117923 19492 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c09c4f15c57044c59d9964440949aaf8: Neither blocks nor log segments found. Creating new log.
I20260809 09:36:57.118911 19492 log.cc:824] T 00000000000000000000000000000000 P c09c4f15c57044c59d9964440949aaf8: Log is configured to *not* fsync() on all Append() calls
I20260809 09:36:57.121300 19492 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c09c4f15c57044c59d9964440949aaf8: No bootstrap required, opened a new log
I20260809 09:36:57.124470 19495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:36:57.119885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50569 (local address 127.0.0.1:39121)
0809 09:36:57.120015 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0809 09:36:57.120019 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 09:36:57.120234 (+ 215us) server_negotiation.cc:407] Connection header received
0809 09:36:57.120357 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:36:57.120359 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:36:57.120388 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:36:57.120455 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:36:57.124009 (+ 3554us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:36:57.124012 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:36:57.124014 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 09:36:57.124026 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:36:57.124052 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 09:36:57.124063 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:36:57.124148 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0809 09:36:57.124289 (+ 141us) server_negotiation.cc:299] Negotiation successful
0809 09:36:57.124324 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260809 09:36:57.125221 19492 raft_consensus.cc:348] T 00000000000000000000000000000000 P c09c4f15c57044c59d9964440949aaf8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c09c4f15c57044c59d9964440949aaf8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39121 } }
I20260809 09:36:57.125478 19492 raft_consensus.cc:374] T 00000000000000000000000000000000 P c09c4f15c57044c59d9964440949aaf8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 09:36:57.125559 19492 raft_consensus.cc:729] T 00000000000000000000000000000000 P c09c4f15c57044c59d9964440949aaf8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c09c4f15c57044c59d9964440949aaf8, State: Initialized, Role: FOLLOWER
I20260809 09:36:57.125977 19492 consensus_queue.cc:260] T 00000000000000000000000000000000 P c09c4f15c57044c59d9964440949aaf8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c09c4f15c57044c59d9964440949aaf8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39121 } }
I20260809 09:36:57.126133 19492 raft_consensus.cc:388] T 00000000000000000000000000000000 P c09c4f15c57044c59d9964440949aaf8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 09:36:57.126220 19492 raft_consensus.cc:482] T 00000000000000000000000000000000 P c09c4f15c57044c59d9964440949aaf8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 09:36:57.126320 19492 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c09c4f15c57044c59d9964440949aaf8 [term 0 FOLLOWER]: Advancing to term 1
I20260809 09:36:57.127259 19492 raft_consensus.cc:504] T 00000000000000000000000000000000 P c09c4f15c57044c59d9964440949aaf8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c09c4f15c57044c59d9964440949aaf8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39121 } }
I20260809 09:36:57.127614 19492 leader_election.cc:302] T 00000000000000000000000000000000 P c09c4f15c57044c59d9964440949aaf8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c09c4f15c57044c59d9964440949aaf8; no voters:
I20260809 09:36:57.127948 19492 leader_election.cc:288] T 00000000000000000000000000000000 P c09c4f15c57044c59d9964440949aaf8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 09:36:57.128083 19504 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c09c4f15c57044c59d9964440949aaf8 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 09:36:57.128345 19504 raft_consensus.cc:686] T 00000000000000000000000000000000 P c09c4f15c57044c59d9964440949aaf8 [term 1 LEADER]: Becoming Leader. State: Replica: c09c4f15c57044c59d9964440949aaf8, State: Running, Role: LEADER
I20260809 09:36:57.128723 19504 consensus_queue.cc:237] T 00000000000000000000000000000000 P c09c4f15c57044c59d9964440949aaf8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c09c4f15c57044c59d9964440949aaf8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39121 } }
I20260809 09:36:57.129262 19492 sys_catalog.cc:564] T 00000000000000000000000000000000 P c09c4f15c57044c59d9964440949aaf8 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 09:36:57.130532 19505 sys_catalog.cc:455] T 00000000000000000000000000000000 P c09c4f15c57044c59d9964440949aaf8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c09c4f15c57044c59d9964440949aaf8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c09c4f15c57044c59d9964440949aaf8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39121 } } }
I20260809 09:36:57.130662 19505 sys_catalog.cc:458] T 00000000000000000000000000000000 P c09c4f15c57044c59d9964440949aaf8 [sys.catalog]: This master's current role is: LEADER
I20260809 09:36:57.131095 19514 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 09:36:57.131050 19506 sys_catalog.cc:455] T 00000000000000000000000000000000 P c09c4f15c57044c59d9964440949aaf8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c09c4f15c57044c59d9964440949aaf8. Latest consensus state: current_term: 1 leader_uuid: "c09c4f15c57044c59d9964440949aaf8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c09c4f15c57044c59d9964440949aaf8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39121 } } }
I20260809 09:36:57.131233 19506 sys_catalog.cc:458] T 00000000000000000000000000000000 P c09c4f15c57044c59d9964440949aaf8 [sys.catalog]: This master's current role is: LEADER
I20260809 09:36:57.134851 19514 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 09:36:57.140503 19514 catalog_manager.cc:1347] Generated new cluster ID: a54965091f3e4cfeaf4a753ba6576dd3
I20260809 09:36:57.140575 19514 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 09:36:57.159735 19514 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 09:36:57.160097 19514 catalog_manager.cc:1504] Loading token signing keys...
I20260809 09:36:57.168875 19514 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c09c4f15c57044c59d9964440949aaf8: Generated new TSK 0
I20260809 09:36:57.169651 19514 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 09:36:57.180490 20982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu with pid 19393
2026-08-09T09:36:57Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1707 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786267941944493-20982-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 09:36:57 dist-test-slave-7kzw krb5kdc[19535](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 09:36:57 dist-test-slave-7kzw krb5kdc[19535](info): set up 2 sockets
Aug 09 09:36:57 dist-test-slave-7kzw krb5kdc[19535](info): commencing operation
krb5kdc: starting...
W20260809 09:36:58.455510 20982 mini_kdc.cc:121] Time spent starting KDC: real 1.242s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786267941944493-20982-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786267941944493-20982-0/krb5kdc/test-user.keytab.
Aug 09 09:36:58 dist-test-slave-7kzw krb5kdc[19535](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268218, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T09:36:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T09:36:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786267941944493-20982-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786267941944493-20982-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 09:36:58.594616 20982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
/tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786267941944493-20982-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786267941944493-20982-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786267941944493-20982-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786267941944493-20982-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38209
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786267941944493-20982-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40921
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38209
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786267941944493-20982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786267941944493-20982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786267941944493-20982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 09:36:58.711211 19550 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 09:36:58.711525 19550 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 09:36:58.711571 19550 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 09:36:58.715881 19550 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 09:36:58.715951 19550 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 09:36:58.715972 19550 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 09:36:58.715989 19550 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 09:36:58.716048 19550 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 09:36:58.721621 19550 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40921
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786267941944493-20982-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786267941944493-20982-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38209
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786267941944493-20982-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38209
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786267941944493-20982-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19550
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786267941944493-20982-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 08:38:55 UTC on ccycloud.quasar-njjleu.root.comops.site
build id 4575935
I20260809 09:36:58.722822 19550 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 09:36:58.723884 19550 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 09:36:58.732800 19560 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:36:58.732787 19562 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:36:58.732791 19564 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:36:58.732946 19550 server_base.cc:1034] running on GCE node
I20260809 09:36:58.733690 19550 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 09:36:58.734841 19550 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 09:36:58.736045 19550 hybrid_clock.cc:648] HybridClock initialized: now 1786268218736027 us; error 30 us; skew 500 ppm
Aug 09 09:36:58 dist-test-slave-7kzw krb5kdc[19535](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268218, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 09:36:58.738891 19550 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 09:36:58.739828 19550 webserver.cc:460] Webserver started at http://127.0.0.1:33255/ using document root <none> and password file <none>
I20260809 09:36:58.740406 19550 fs_manager.cc:359] Metadata directory not provided
I20260809 09:36:58.740504 19550 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 09:36:58.740762 19550 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 09:36:58.742974 19550 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786267941944493-20982-0/minicluster-data/master-0/data/instance:
uuid: "d64ecff588fc4176ab15a2c542522a9b"
format_stamp: "Formatted at 2026-08-09 09:36:58 on dist-test-slave-7kzw"
server_key: "bd780215bd455eeb5974f4b7ab136103"
server_key_iv: "8bcc3c402b1e839383f105e28db6fe6c"
server_key_version: "encryptionkey@0"
I20260809 09:36:58.743643 19550 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786267941944493-20982-0/minicluster-data/master-0/wal/instance:
uuid: "d64ecff588fc4176ab15a2c542522a9b"
format_stamp: "Formatted at 2026-08-09 09:36:58 on dist-test-slave-7kzw"
server_key: "bd780215bd455eeb5974f4b7ab136103"
server_key_iv: "8bcc3c402b1e839383f105e28db6fe6c"
server_key_version: "encryptionkey@0"
I20260809 09:36:58.747602 19550 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260809 09:36:58.750123 19575 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 09:36:58.751418 19550 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260809 09:36:58.751580 19550 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786267941944493-20982-0/minicluster-data/master-0/data,/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786267941944493-20982-0/minicluster-data/master-0/wal
uuid: "d64ecff588fc4176ab15a2c542522a9b"
format_stamp: "Formatted at 2026-08-09 09:36:58 on dist-test-slave-7kzw"
server_key: "bd780215bd455eeb5974f4b7ab136103"
server_key_iv: "8bcc3c402b1e839383f105e28db6fe6c"
server_key_version: "encryptionkey@0"
I20260809 09:36:58.751700 19550 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786267941944493-20982-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786267941944493-20982-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786267941944493-20982-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 09:36:58.767668 19550 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 09:36:58.770545 19550 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 09:36:58.770761 19550 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 09:36:58.778110 19550 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38209
I20260809 09:36:58.778138 19661 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38209 every 8 connection(s)
I20260809 09:36:58.779098 19550 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786267941944493-20982-0/minicluster-data/master-0/data/info.pb
I20260809 09:36:58.781920 19662 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 09:36:58.782183 20982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu as pid 19550
I20260809 09:36:58.782308 20982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786267941944493-20982-0/minicluster-data/master-0/wal/instance
I20260809 09:36:58.782645 20982 external_mini_cluster.cc:1428] Setting key 9752283f976f74c1735ede9d81394b29
I20260809 09:36:58.788445 19662 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d64ecff588fc4176ab15a2c542522a9b: Bootstrap starting.
I20260809 09:36:58.790815 19662 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d64ecff588fc4176ab15a2c542522a9b: Neither blocks nor log segments found. Creating new log.
Aug 09 09:36:58 dist-test-slave-7kzw krb5kdc[19535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268218, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260809 09:36:58.791779 19662 log.cc:824] T 00000000000000000000000000000000 P d64ecff588fc4176ab15a2c542522a9b: Log is configured to *not* fsync() on all Append() calls
I20260809 09:36:58.794098 19662 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d64ecff588fc4176ab15a2c542522a9b: No bootstrap required, opened a new log
I20260809 09:36:58.797343 19665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:36:58.784106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.136:48209 (local address 127.0.0.1:38209)
0809 09:36:58.784546 (+ 440us) server_negotiation.cc:206] Beginning negotiation
0809 09:36:58.784561 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 09:36:58.784595 (+ 34us) server_negotiation.cc:407] Connection header received
0809 09:36:58.785272 (+ 677us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:36:58.785288 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:36:58.785610 (+ 322us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:36:58.785964 (+ 354us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:36:58.786890 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:36:58.787860 (+ 970us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:36:58.788819 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:36:58.789282 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:36:58.789459 (+ 177us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:36:58.792465 (+ 3006us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:36:58.792471 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:36:58.792489 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:36:58.792523 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:36:58.794323 (+ 1800us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:36:58.794929 (+ 606us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:36:58.794934 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:36:58.794938 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:36:58.795047 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:36:58.795428 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:36:58.795430 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:36:58.795432 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:36:58.795807 (+ 375us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:36:58.795975 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0809 09:36:58.796340 (+ 365us) server_negotiation.cc:299] Negotiation successful
0809 09:36:58.796584 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":288,"thread_start_us":123,"threads_started":1}
I20260809 09:36:58.798233 19662 raft_consensus.cc:348] T 00000000000000000000000000000000 P d64ecff588fc4176ab15a2c542522a9b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d64ecff588fc4176ab15a2c542522a9b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38209 } }
I20260809 09:36:58.798535 19662 raft_consensus.cc:374] T 00000000000000000000000000000000 P d64ecff588fc4176ab15a2c542522a9b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 09:36:58.798601 19662 raft_consensus.cc:729] T 00000000000000000000000000000000 P d64ecff588fc4176ab15a2c542522a9b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d64ecff588fc4176ab15a2c542522a9b, State: Initialized, Role: FOLLOWER
I20260809 09:36:58.799058 19662 consensus_queue.cc:260] T 00000000000000000000000000000000 P d64ecff588fc4176ab15a2c542522a9b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d64ecff588fc4176ab15a2c542522a9b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38209 } }
I20260809 09:36:58.799229 19662 raft_consensus.cc:388] T 00000000000000000000000000000000 P d64ecff588fc4176ab15a2c542522a9b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 09:36:58.799314 19662 raft_consensus.cc:482] T 00000000000000000000000000000000 P d64ecff588fc4176ab15a2c542522a9b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 09:36:58.799418 19662 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d64ecff588fc4176ab15a2c542522a9b [term 0 FOLLOWER]: Advancing to term 1
I20260809 09:36:58.800536 19662 raft_consensus.cc:504] T 00000000000000000000000000000000 P d64ecff588fc4176ab15a2c542522a9b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d64ecff588fc4176ab15a2c542522a9b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38209 } }
I20260809 09:36:58.800590 20982 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 09:36:58.800882 19662 leader_election.cc:302] T 00000000000000000000000000000000 P d64ecff588fc4176ab15a2c542522a9b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d64ecff588fc4176ab15a2c542522a9b; no voters:
I20260809 09:36:58.801216 19662 leader_election.cc:288] T 00000000000000000000000000000000 P d64ecff588fc4176ab15a2c542522a9b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 09:36:58.801291 19667 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d64ecff588fc4176ab15a2c542522a9b [term 1 FOLLOWER]: Leader election won for term 1
I20260809 09:36:58.801494 19667 raft_consensus.cc:686] T 00000000000000000000000000000000 P d64ecff588fc4176ab15a2c542522a9b [term 1 LEADER]: Becoming Leader. State: Replica: d64ecff588fc4176ab15a2c542522a9b, State: Running, Role: LEADER
I20260809 09:36:58.801843 19667 consensus_queue.cc:237] T 00000000000000000000000000000000 P d64ecff588fc4176ab15a2c542522a9b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d64ecff588fc4176ab15a2c542522a9b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38209 } }
I20260809 09:36:58.802543 19662 sys_catalog.cc:564] T 00000000000000000000000000000000 P d64ecff588fc4176ab15a2c542522a9b [sys.catalog]: configured and running, proceeding with master startup.
I20260809 09:36:58.803823 19669 sys_catalog.cc:455] T 00000000000000000000000000000000 P d64ecff588fc4176ab15a2c542522a9b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d64ecff588fc4176ab15a2c542522a9b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d64ecff588fc4176ab15a2c542522a9b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38209 } } }
I20260809 09:36:58.803970 19669 sys_catalog.cc:458] T 00000000000000000000000000000000 P d64ecff588fc4176ab15a2c542522a9b [sys.catalog]: This master's current role is: LEADER
I20260809 09:36:58.804436 19691 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 09:36:58.805385 19670 sys_catalog.cc:455] T 00000000000000000000000000000000 P d64ecff588fc4176ab15a2c542522a9b [sys.catalog]: SysCatalogTable state changed. Reason: New leader d64ecff588fc4176ab15a2c542522a9b. Latest consensus state: current_term: 1 leader_uuid: "d64ecff588fc4176ab15a2c542522a9b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d64ecff588fc4176ab15a2c542522a9b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38209 } } }
I20260809 09:36:58.805562 19670 sys_catalog.cc:458] T 00000000000000000000000000000000 P d64ecff588fc4176ab15a2c542522a9b [sys.catalog]: This master's current role is: LEADER
I20260809 09:36:58.807749 19691 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 09:36:58.813251 19665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:36:58.802997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.136:33177 (local address 127.0.0.1:38209)
0809 09:36:58.803124 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0809 09:36:58.803128 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 09:36:58.804214 (+ 1086us) server_negotiation.cc:407] Connection header received
0809 09:36:58.804251 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:36:58.804253 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:36:58.804292 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:36:58.804392 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:36:58.805324 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:36:58.806007 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:36:58.807197 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:36:58.807562 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:36:58.807674 (+ 112us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:36:58.809565 (+ 1891us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:36:58.809569 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:36:58.809572 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:36:58.809598 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:36:58.811132 (+ 1534us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:36:58.811712 (+ 580us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:36:58.811715 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:36:58.811717 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:36:58.811812 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:36:58.812207 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:36:58.812211 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:36:58.812212 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:36:58.812448 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:36:58.812604 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0809 09:36:58.812969 (+ 365us) server_negotiation.cc:299] Negotiation successful
0809 09:36:58.813075 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260809 09:36:58.815761 19691 catalog_manager.cc:1347] Generated new cluster ID: e03d94f9a5044c718db56366138c53c3
I20260809 09:36:58.815824 19691 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 09:36:58.840377 19691 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 09:36:58.841326 19691 catalog_manager.cc:1504] Loading token signing keys...
I20260809 09:36:58.857280 19691 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d64ecff588fc4176ab15a2c542522a9b: Generated new TSK 0
I20260809 09:36:58.858274 19691 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 09:36:58.873860 20982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu with pid 19550
W20260809 09:36:58.891448 19532 connection.cc:535] client connection to 127.0.0.1:38209 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:38209): Connection reset by peer (error 104)
2026-08-09T09:36:58Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1693 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786267941944493-20982-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 09:36:58 dist-test-slave-7kzw krb5kdc[19704](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 09:36:58 dist-test-slave-7kzw krb5kdc[19704](info): set up 2 sockets
Aug 09 09:36:58 dist-test-slave-7kzw krb5kdc[19704](info): commencing operation
krb5kdc: starting...
W20260809 09:37:00.158794 20982 mini_kdc.cc:121] Time spent starting KDC: real 1.252s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786267941944493-20982-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786267941944493-20982-0/krb5kdc/test-user.keytab.
Aug 09 09:37:00 dist-test-slave-7kzw krb5kdc[19704](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268220, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T09:37:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T09:37:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786267941944493-20982-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786267941944493-20982-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 09:37:00.315238 20982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
/tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786267941944493-20982-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786267941944493-20982-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786267941944493-20982-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786267941944493-20982-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34601
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786267941944493-20982-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43669
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34601
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786267941944493-20982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786267941944493-20982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786267941944493-20982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 09:37:00.435374 19728 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 09:37:00.435719 19728 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 09:37:00.435797 19728 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 09:37:00.439818 19728 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 09:37:00.439949 19728 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 09:37:00.439989 19728 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 09:37:00.440028 19728 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 09:37:00.440058 19728 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 09:37:00.446868 19728 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43669
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786267941944493-20982-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786267941944493-20982-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34601
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786267941944493-20982-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34601
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786267941944493-20982-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19728
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786267941944493-20982-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 08:38:55 UTC on ccycloud.quasar-njjleu.root.comops.site
build id 4575935
I20260809 09:37:00.448227 19728 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 09:37:00.449412 19728 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 09:37:00.458837 19739 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:37:00.458844 19736 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:37:00.458840 19737 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:37:00.459398 19728 server_base.cc:1034] running on GCE node
I20260809 09:37:00.460017 19728 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 09:37:00.461190 19728 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 09:37:00.462371 19728 hybrid_clock.cc:648] HybridClock initialized: now 1786268220462347 us; error 34 us; skew 500 ppm
Aug 09 09:37:00 dist-test-slave-7kzw krb5kdc[19704](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268220, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 09:37:00.465312 19728 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 09:37:00.466223 19728 webserver.cc:460] Webserver started at http://127.0.0.1:46285/ using document root <none> and password file <none>
I20260809 09:37:00.466859 19728 fs_manager.cc:359] Metadata directory not provided
I20260809 09:37:00.466926 19728 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 09:37:00.467098 19728 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 09:37:00.469199 19728 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786267941944493-20982-0/minicluster-data/master-0/data/instance:
uuid: "6d9f1be558ab45fead670c14ad29e502"
format_stamp: "Formatted at 2026-08-09 09:37:00 on dist-test-slave-7kzw"
server_key: "7e9434c885f6d4aa74bc69ea1d58bd57"
server_key_iv: "5c1bd8aae19569046f69c2a3eafea6e4"
server_key_version: "encryptionkey@0"
I20260809 09:37:00.469738 19728 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786267941944493-20982-0/minicluster-data/master-0/wal/instance:
uuid: "6d9f1be558ab45fead670c14ad29e502"
format_stamp: "Formatted at 2026-08-09 09:37:00 on dist-test-slave-7kzw"
server_key: "7e9434c885f6d4aa74bc69ea1d58bd57"
server_key_iv: "5c1bd8aae19569046f69c2a3eafea6e4"
server_key_version: "encryptionkey@0"
I20260809 09:37:00.473672 19728 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.003s
I20260809 09:37:00.476469 19752 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 09:37:00.477943 19728 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 09:37:00.478065 19728 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786267941944493-20982-0/minicluster-data/master-0/data,/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786267941944493-20982-0/minicluster-data/master-0/wal
uuid: "6d9f1be558ab45fead670c14ad29e502"
format_stamp: "Formatted at 2026-08-09 09:37:00 on dist-test-slave-7kzw"
server_key: "7e9434c885f6d4aa74bc69ea1d58bd57"
server_key_iv: "5c1bd8aae19569046f69c2a3eafea6e4"
server_key_version: "encryptionkey@0"
I20260809 09:37:00.478166 19728 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786267941944493-20982-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786267941944493-20982-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786267941944493-20982-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 09:37:00.502563 19728 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 09:37:00.506451 19728 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 09:37:00.506752 19728 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 09:37:00.514851 19728 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34601
I20260809 09:37:00.514955 19838 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34601 every 8 connection(s)
I20260809 09:37:00.515961 19728 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786267941944493-20982-0/minicluster-data/master-0/data/info.pb
I20260809 09:37:00.519043 19839 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 09:37:00.524962 20982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu as pid 19728
I20260809 09:37:00.525149 20982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786267941944493-20982-0/minicluster-data/master-0/wal/instance
I20260809 09:37:00.525243 19839 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6d9f1be558ab45fead670c14ad29e502: Bootstrap starting.
I20260809 09:37:00.525485 20982 external_mini_cluster.cc:1428] Setting key 54be1ee2afdcfe805e9643c03772977d
I20260809 09:37:00.528072 19839 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6d9f1be558ab45fead670c14ad29e502: Neither blocks nor log segments found. Creating new log.
I20260809 09:37:00.528981 19839 log.cc:824] T 00000000000000000000000000000000 P 6d9f1be558ab45fead670c14ad29e502: Log is configured to *not* fsync() on all Append() calls
I20260809 09:37:00.531244 19839 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6d9f1be558ab45fead670c14ad29e502: No bootstrap required, opened a new log
I20260809 09:37:00.535252 19843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:37:00.526930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.136:55475 (local address 127.0.0.1:34601)
0809 09:37:00.527376 (+ 446us) server_negotiation.cc:206] Beginning negotiation
0809 09:37:00.527400 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0809 09:37:00.527437 (+ 37us) server_negotiation.cc:407] Connection header received
0809 09:37:00.528083 (+ 646us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:37:00.528099 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:37:00.528509 (+ 410us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:37:00.528815 (+ 306us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:37:00.529231 (+ 416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:37:00.530207 (+ 976us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:37:00.531489 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:37:00.532030 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:37:00.532210 (+ 180us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:37:00.533232 (+ 1022us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:37:00.533238 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:37:00.533252 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 09:37:00.533686 (+ 434us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:37:00.533748 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 09:37:00.533773 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:37:00.533889 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0809 09:37:00.534154 (+ 265us) server_negotiation.cc:299] Negotiation successful
0809 09:37:00.534349 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":277,"thread_start_us":151,"threads_started":1}
I20260809 09:37:00.535629 19839 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6d9f1be558ab45fead670c14ad29e502 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d9f1be558ab45fead670c14ad29e502" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34601 } }
I20260809 09:37:00.535856 19839 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6d9f1be558ab45fead670c14ad29e502 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 09:37:00.535918 19839 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6d9f1be558ab45fead670c14ad29e502 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d9f1be558ab45fead670c14ad29e502, State: Initialized, Role: FOLLOWER
I20260809 09:37:00.536356 19839 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6d9f1be558ab45fead670c14ad29e502 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d9f1be558ab45fead670c14ad29e502" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34601 } }
I20260809 09:37:00.536520 19839 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6d9f1be558ab45fead670c14ad29e502 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 09:37:00.536625 19839 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6d9f1be558ab45fead670c14ad29e502 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 09:37:00.536760 19839 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6d9f1be558ab45fead670c14ad29e502 [term 0 FOLLOWER]: Advancing to term 1
I20260809 09:37:00.537827 19839 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6d9f1be558ab45fead670c14ad29e502 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d9f1be558ab45fead670c14ad29e502" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34601 } }
I20260809 09:37:00.538228 19839 leader_election.cc:302] T 00000000000000000000000000000000 P 6d9f1be558ab45fead670c14ad29e502 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6d9f1be558ab45fead670c14ad29e502; no voters:
I20260809 09:37:00.538770 19839 leader_election.cc:288] T 00000000000000000000000000000000 P 6d9f1be558ab45fead670c14ad29e502 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 09:37:00.538911 19847 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6d9f1be558ab45fead670c14ad29e502 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 09:37:00.539211 19847 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6d9f1be558ab45fead670c14ad29e502 [term 1 LEADER]: Becoming Leader. State: Replica: 6d9f1be558ab45fead670c14ad29e502, State: Running, Role: LEADER
I20260809 09:37:00.539255 20982 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 09:37:00.539582 19847 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6d9f1be558ab45fead670c14ad29e502 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d9f1be558ab45fead670c14ad29e502" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34601 } }
I20260809 09:37:00.540246 19839 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6d9f1be558ab45fead670c14ad29e502 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 09:37:00.541337 19849 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6d9f1be558ab45fead670c14ad29e502 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6d9f1be558ab45fead670c14ad29e502" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d9f1be558ab45fead670c14ad29e502" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34601 } } }
I20260809 09:37:00.541473 19849 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6d9f1be558ab45fead670c14ad29e502 [sys.catalog]: This master's current role is: LEADER
I20260809 09:37:00.541759 19849 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6d9f1be558ab45fead670c14ad29e502 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6d9f1be558ab45fead670c14ad29e502. Latest consensus state: current_term: 1 leader_uuid: "6d9f1be558ab45fead670c14ad29e502" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d9f1be558ab45fead670c14ad29e502" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34601 } } }
I20260809 09:37:00.541844 19849 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6d9f1be558ab45fead670c14ad29e502 [sys.catalog]: This master's current role is: LEADER
I20260809 09:37:00.541901 19865 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 09:37:00.545207 19865 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 09:37:00.547298 19843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:37:00.541927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.136:48487 (local address 127.0.0.1:34601)
0809 09:37:00.542065 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0809 09:37:00.542070 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 09:37:00.542548 (+ 478us) server_negotiation.cc:407] Connection header received
0809 09:37:00.542672 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:37:00.542675 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:37:00.542731 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:37:00.542818 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:37:00.543205 (+ 387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:37:00.543885 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:37:00.545254 (+ 1369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:37:00.545543 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:37:00.545657 (+ 114us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:37:00.546694 (+ 1037us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:37:00.546699 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:37:00.546702 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 09:37:00.546717 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:37:00.546759 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 09:37:00.546774 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:37:00.546886 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0809 09:37:00.547075 (+ 189us) server_negotiation.cc:299] Negotiation successful
0809 09:37:00.547116 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260809 09:37:00.552135 19865 catalog_manager.cc:1347] Generated new cluster ID: 1c1b292218af48b1aa3ae5db828ec95c
I20260809 09:37:00.552263 19865 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 09:37:00.566988 19865 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 09:37:00.568039 19865 catalog_manager.cc:1504] Loading token signing keys...
I20260809 09:37:00.580490 19865 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6d9f1be558ab45fead670c14ad29e502: Generated new TSK 0
I20260809 09:37:00.581383 19865 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 09:37:00.587116 20982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu with pid 19728
W20260809 09:37:00.603197 19702 connection.cc:535] client connection to 127.0.0.1:34601 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:34601): Connection reset by peer (error 104)
2026-08-09T09:37:00Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1711 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786267941944493-20982-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 09:37:00 dist-test-slave-7kzw krb5kdc[19892](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 09:37:00 dist-test-slave-7kzw krb5kdc[19892](info): set up 2 sockets
Aug 09 09:37:00 dist-test-slave-7kzw krb5kdc[19892](info): commencing operation
krb5kdc: starting...
W20260809 09:37:01.932839 20982 mini_kdc.cc:121] Time spent starting KDC: real 1.315s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786267941944493-20982-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786267941944493-20982-0/krb5kdc/test-user.keytab.
Aug 09 09:37:02 dist-test-slave-7kzw krb5kdc[19892](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268222, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T09:37:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T09:37:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786267941944493-20982-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786267941944493-20982-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 09:37:02.078770 20982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
/tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786267941944493-20982-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786267941944493-20982-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786267941944493-20982-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786267941944493-20982-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41683
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786267941944493-20982-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33095
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41683
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786267941944493-20982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786267941944493-20982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786267941944493-20982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 09:37:02.194731 19907 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 09:37:02.195073 19907 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 09:37:02.195164 19907 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 09:37:02.199227 19907 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 09:37:02.199319 19907 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 09:37:02.199355 19907 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 09:37:02.199419 19907 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 09:37:02.199458 19907 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 09:37:02.205211 19907 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33095
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786267941944493-20982-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786267941944493-20982-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41683
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786267941944493-20982-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41683
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786267941944493-20982-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19907
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786267941944493-20982-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 08:38:55 UTC on ccycloud.quasar-njjleu.root.comops.site
build id 4575935
I20260809 09:37:02.206609 19907 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 09:37:02.207821 19907 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 09:37:02.217975 19917 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:37:02.217975 19914 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:37:02.218009 19915 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:37:02.218421 19907 server_base.cc:1034] running on GCE node
I20260809 09:37:02.219130 19907 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 09:37:02.220377 19907 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 09:37:02.221601 19907 hybrid_clock.cc:648] HybridClock initialized: now 1786268222221571 us; error 43 us; skew 500 ppm
Aug 09 09:37:02 dist-test-slave-7kzw krb5kdc[19892](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268222, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 09:37:02.229283 19907 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 09:37:02.230291 19907 webserver.cc:460] Webserver started at http://127.0.0.1:33903/ using document root <none> and password file <none>
I20260809 09:37:02.230957 19907 fs_manager.cc:359] Metadata directory not provided
I20260809 09:37:02.231105 19907 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 09:37:02.231359 19907 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 09:37:02.233490 19907 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786267941944493-20982-0/minicluster-data/master-0/data/instance:
uuid: "80b0dfa9d6fe417ba36dca45edc63048"
format_stamp: "Formatted at 2026-08-09 09:37:02 on dist-test-slave-7kzw"
server_key: "82af4572b224bd2ba280dff532edffad"
server_key_iv: "46eda231326d5cd75a5b79b4c8201ae8"
server_key_version: "encryptionkey@0"
I20260809 09:37:02.234082 19907 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786267941944493-20982-0/minicluster-data/master-0/wal/instance:
uuid: "80b0dfa9d6fe417ba36dca45edc63048"
format_stamp: "Formatted at 2026-08-09 09:37:02 on dist-test-slave-7kzw"
server_key: "82af4572b224bd2ba280dff532edffad"
server_key_iv: "46eda231326d5cd75a5b79b4c8201ae8"
server_key_version: "encryptionkey@0"
I20260809 09:37:02.237922 19907 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260809 09:37:02.240756 19924 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 09:37:02.242142 19907 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 09:37:02.242321 19907 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786267941944493-20982-0/minicluster-data/master-0/data,/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786267941944493-20982-0/minicluster-data/master-0/wal
uuid: "80b0dfa9d6fe417ba36dca45edc63048"
format_stamp: "Formatted at 2026-08-09 09:37:02 on dist-test-slave-7kzw"
server_key: "82af4572b224bd2ba280dff532edffad"
server_key_iv: "46eda231326d5cd75a5b79b4c8201ae8"
server_key_version: "encryptionkey@0"
I20260809 09:37:02.242453 19907 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786267941944493-20982-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786267941944493-20982-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786267941944493-20982-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 09:37:02.247723 19907 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 09:37:02.251320 19907 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 09:37:02.251570 19907 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 09:37:02.259994 19907 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41683
I20260809 09:37:02.260035 20003 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41683 every 8 connection(s)
I20260809 09:37:02.261055 19907 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786267941944493-20982-0/minicluster-data/master-0/data/info.pb
I20260809 09:37:02.264019 20004 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 09:37:02.266819 20982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu as pid 19907
I20260809 09:37:02.266960 20982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786267941944493-20982-0/minicluster-data/master-0/wal/instance
I20260809 09:37:02.267292 20982 external_mini_cluster.cc:1428] Setting key a8856f58980e970188aaf5df18c7d587
I20260809 09:37:02.270259 20004 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 80b0dfa9d6fe417ba36dca45edc63048: Bootstrap starting.
I20260809 09:37:02.272746 20007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:37:02.268763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.136:60153 (local address 127.0.0.1:41683)
0809 09:37:02.269234 (+ 471us) server_negotiation.cc:206] Beginning negotiation
0809 09:37:02.269249 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 09:37:02.269286 (+ 37us) server_negotiation.cc:407] Connection header received
0809 09:37:02.269891 (+ 605us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:37:02.269906 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:37:02.270190 (+ 284us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:37:02.270554 (+ 364us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:37:02.271558 (+ 1004us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:37:02.271563 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:37:02.271575 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 09:37:02.271586 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:37:02.271648 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 09:37:02.271668 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:37:02.271829 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0809 09:37:02.271902 (+ 73us) server_negotiation.cc:299] Negotiation successful
0809 09:37:02.272067 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":296,"thread_start_us":127,"threads_started":1}
I20260809 09:37:02.272846 20004 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 80b0dfa9d6fe417ba36dca45edc63048: Neither blocks nor log segments found. Creating new log.
I20260809 09:37:02.273778 20004 log.cc:824] T 00000000000000000000000000000000 P 80b0dfa9d6fe417ba36dca45edc63048: Log is configured to *not* fsync() on all Append() calls
I20260809 09:37:02.275990 20982 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 09:37:02.276144 20004 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 80b0dfa9d6fe417ba36dca45edc63048: No bootstrap required, opened a new log
I20260809 09:37:02.280270 20007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:37:02.278224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.136:56941 (local address 127.0.0.1:41683)
0809 09:37:02.278365 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0809 09:37:02.278369 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 09:37:02.278610 (+ 241us) server_negotiation.cc:407] Connection header received
0809 09:37:02.278819 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:37:02.278825 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:37:02.278903 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:37:02.279043 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:37:02.279907 (+ 864us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:37:02.279909 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:37:02.279911 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 09:37:02.279922 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:37:02.279947 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 09:37:02.279957 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:37:02.280027 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0809 09:37:02.280139 (+ 112us) server_negotiation.cc:299] Negotiation successful
0809 09:37:02.280172 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 09:37:02.280658 20004 raft_consensus.cc:348] T 00000000000000000000000000000000 P 80b0dfa9d6fe417ba36dca45edc63048 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80b0dfa9d6fe417ba36dca45edc63048" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41683 } }
I20260809 09:37:02.280929 20004 raft_consensus.cc:374] T 00000000000000000000000000000000 P 80b0dfa9d6fe417ba36dca45edc63048 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 09:37:02.280990 20004 raft_consensus.cc:729] T 00000000000000000000000000000000 P 80b0dfa9d6fe417ba36dca45edc63048 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80b0dfa9d6fe417ba36dca45edc63048, State: Initialized, Role: FOLLOWER
I20260809 09:37:02.281466 20004 consensus_queue.cc:260] T 00000000000000000000000000000000 P 80b0dfa9d6fe417ba36dca45edc63048 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80b0dfa9d6fe417ba36dca45edc63048" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41683 } }
I20260809 09:37:02.281630 20004 raft_consensus.cc:388] T 00000000000000000000000000000000 P 80b0dfa9d6fe417ba36dca45edc63048 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 09:37:02.281706 20004 raft_consensus.cc:482] T 00000000000000000000000000000000 P 80b0dfa9d6fe417ba36dca45edc63048 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 09:37:02.281811 20004 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 80b0dfa9d6fe417ba36dca45edc63048 [term 0 FOLLOWER]: Advancing to term 1
I20260809 09:37:02.282891 20004 raft_consensus.cc:504] T 00000000000000000000000000000000 P 80b0dfa9d6fe417ba36dca45edc63048 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80b0dfa9d6fe417ba36dca45edc63048" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41683 } }
I20260809 09:37:02.283258 20004 leader_election.cc:302] T 00000000000000000000000000000000 P 80b0dfa9d6fe417ba36dca45edc63048 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 80b0dfa9d6fe417ba36dca45edc63048; no voters:
I20260809 09:37:02.283597 20004 leader_election.cc:288] T 00000000000000000000000000000000 P 80b0dfa9d6fe417ba36dca45edc63048 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 09:37:02.283790 20020 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 80b0dfa9d6fe417ba36dca45edc63048 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 09:37:02.284139 20020 raft_consensus.cc:686] T 00000000000000000000000000000000 P 80b0dfa9d6fe417ba36dca45edc63048 [term 1 LEADER]: Becoming Leader. State: Replica: 80b0dfa9d6fe417ba36dca45edc63048, State: Running, Role: LEADER
I20260809 09:37:02.284545 20020 consensus_queue.cc:237] T 00000000000000000000000000000000 P 80b0dfa9d6fe417ba36dca45edc63048 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80b0dfa9d6fe417ba36dca45edc63048" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41683 } }
I20260809 09:37:02.284919 20004 sys_catalog.cc:564] T 00000000000000000000000000000000 P 80b0dfa9d6fe417ba36dca45edc63048 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 09:37:02.286309 20023 sys_catalog.cc:455] T 00000000000000000000000000000000 P 80b0dfa9d6fe417ba36dca45edc63048 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 80b0dfa9d6fe417ba36dca45edc63048. Latest consensus state: current_term: 1 leader_uuid: "80b0dfa9d6fe417ba36dca45edc63048" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80b0dfa9d6fe417ba36dca45edc63048" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41683 } } }
I20260809 09:37:02.286448 20023 sys_catalog.cc:458] T 00000000000000000000000000000000 P 80b0dfa9d6fe417ba36dca45edc63048 [sys.catalog]: This master's current role is: LEADER
I20260809 09:37:02.286897 20035 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 09:37:02.286862 20021 sys_catalog.cc:455] T 00000000000000000000000000000000 P 80b0dfa9d6fe417ba36dca45edc63048 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "80b0dfa9d6fe417ba36dca45edc63048" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80b0dfa9d6fe417ba36dca45edc63048" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41683 } } }
I20260809 09:37:02.286952 20021 sys_catalog.cc:458] T 00000000000000000000000000000000 P 80b0dfa9d6fe417ba36dca45edc63048 [sys.catalog]: This master's current role is: LEADER
I20260809 09:37:02.289968 20035 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 09:37:02.295149 20035 catalog_manager.cc:1347] Generated new cluster ID: d46678bd14454dd2a1faa2b980911725
I20260809 09:37:02.295217 20035 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 09:37:02.301286 20035 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 09:37:02.301622 20035 catalog_manager.cc:1504] Loading token signing keys...
I20260809 09:37:02.324996 20035 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 80b0dfa9d6fe417ba36dca45edc63048: Generated new TSK 0
I20260809 09:37:02.325788 20035 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 09:37:02.338225 20982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu with pid 19907
2026-08-09T09:37:02Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1743 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786267941944493-20982-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 09:37:02 dist-test-slave-7kzw krb5kdc[20052](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 09:37:02 dist-test-slave-7kzw krb5kdc[20052](info): set up 2 sockets
Aug 09 09:37:02 dist-test-slave-7kzw krb5kdc[20052](info): commencing operation
krb5kdc: starting...
W20260809 09:37:03.570785 20982 mini_kdc.cc:121] Time spent starting KDC: real 1.209s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786267941944493-20982-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786267941944493-20982-0/krb5kdc/test-user.keytab.
Aug 09 09:37:03 dist-test-slave-7kzw krb5kdc[20052](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268223, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T09:37:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T09:37:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786267941944493-20982-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786267941944493-20982-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 09:37:03.706251 20982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
/tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786267941944493-20982-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786267941944493-20982-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786267941944493-20982-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786267941944493-20982-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35749
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786267941944493-20982-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40293
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35749
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786267941944493-20982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786267941944493-20982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786267941944493-20982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 09:37:03.827634 20076 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 09:37:03.827929 20076 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 09:37:03.828037 20076 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 09:37:03.834173 20076 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 09:37:03.834333 20076 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 09:37:03.834381 20076 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 09:37:03.834419 20076 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 09:37:03.834509 20076 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 09:37:03.841387 20076 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40293
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786267941944493-20982-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786267941944493-20982-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35749
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786267941944493-20982-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35749
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786267941944493-20982-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20076
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786267941944493-20982-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 08:38:55 UTC on ccycloud.quasar-njjleu.root.comops.site
build id 4575935
I20260809 09:37:03.842823 20076 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 09:37:03.843884 20076 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 09:37:03.852813 20083 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:37:03.852823 20084 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:37:03.852813 20088 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:37:03.853046 20076 server_base.cc:1034] running on GCE node
I20260809 09:37:03.853775 20076 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 09:37:03.854981 20076 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 09:37:03.856189 20076 hybrid_clock.cc:648] HybridClock initialized: now 1786268223856171 us; error 30 us; skew 500 ppm
Aug 09 09:37:03 dist-test-slave-7kzw krb5kdc[20052](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268223, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 09:37:03.859217 20076 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 09:37:03.860190 20076 webserver.cc:460] Webserver started at http://127.0.0.1:39563/ using document root <none> and password file <none>
I20260809 09:37:03.860738 20076 fs_manager.cc:359] Metadata directory not provided
I20260809 09:37:03.860802 20076 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 09:37:03.860971 20076 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 09:37:03.863235 20076 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786267941944493-20982-0/minicluster-data/master-0/data/instance:
uuid: "70d7b8f0a9d34ae48a61a19cfe8cb141"
format_stamp: "Formatted at 2026-08-09 09:37:03 on dist-test-slave-7kzw"
server_key: "1f7f34d70003545b0a76a5742321a798"
server_key_iv: "acba746b4748aaa2d17a5c78b0ad1d81"
server_key_version: "encryptionkey@0"
I20260809 09:37:03.863838 20076 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786267941944493-20982-0/minicluster-data/master-0/wal/instance:
uuid: "70d7b8f0a9d34ae48a61a19cfe8cb141"
format_stamp: "Formatted at 2026-08-09 09:37:03 on dist-test-slave-7kzw"
server_key: "1f7f34d70003545b0a76a5742321a798"
server_key_iv: "acba746b4748aaa2d17a5c78b0ad1d81"
server_key_version: "encryptionkey@0"
I20260809 09:37:03.867831 20076 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.004s
I20260809 09:37:03.870530 20102 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 09:37:03.871929 20076 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 09:37:03.872046 20076 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786267941944493-20982-0/minicluster-data/master-0/data,/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786267941944493-20982-0/minicluster-data/master-0/wal
uuid: "70d7b8f0a9d34ae48a61a19cfe8cb141"
format_stamp: "Formatted at 2026-08-09 09:37:03 on dist-test-slave-7kzw"
server_key: "1f7f34d70003545b0a76a5742321a798"
server_key_iv: "acba746b4748aaa2d17a5c78b0ad1d81"
server_key_version: "encryptionkey@0"
I20260809 09:37:03.872200 20076 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786267941944493-20982-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786267941944493-20982-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786267941944493-20982-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 09:37:03.886655 20076 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 09:37:03.889808 20076 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 09:37:03.890021 20076 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 09:37:03.897652 20076 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35749
I20260809 09:37:03.897677 20184 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35749 every 8 connection(s)
I20260809 09:37:03.898707 20076 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786267941944493-20982-0/minicluster-data/master-0/data/info.pb
I20260809 09:37:03.901602 20186 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 09:37:03.904444 20982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu as pid 20076
I20260809 09:37:03.904635 20982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786267941944493-20982-0/minicluster-data/master-0/wal/instance
I20260809 09:37:03.904980 20982 external_mini_cluster.cc:1428] Setting key 35551efd2a297e71205c8f5e090b8db2
I20260809 09:37:03.908464 20186 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 70d7b8f0a9d34ae48a61a19cfe8cb141: Bootstrap starting.
I20260809 09:37:03.911237 20186 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 70d7b8f0a9d34ae48a61a19cfe8cb141: Neither blocks nor log segments found. Creating new log.
I20260809 09:37:03.912122 20186 log.cc:824] T 00000000000000000000000000000000 P 70d7b8f0a9d34ae48a61a19cfe8cb141: Log is configured to *not* fsync() on all Append() calls
I20260809 09:37:03.914079 20186 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 70d7b8f0a9d34ae48a61a19cfe8cb141: No bootstrap required, opened a new log
Aug 09 09:37:03 dist-test-slave-7kzw krb5kdc[20052](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268223, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260809 09:37:03.917601 20186 raft_consensus.cc:348] T 00000000000000000000000000000000 P 70d7b8f0a9d34ae48a61a19cfe8cb141 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70d7b8f0a9d34ae48a61a19cfe8cb141" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35749 } }
I20260809 09:37:03.917819 20186 raft_consensus.cc:374] T 00000000000000000000000000000000 P 70d7b8f0a9d34ae48a61a19cfe8cb141 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 09:37:03.917855 20186 raft_consensus.cc:729] T 00000000000000000000000000000000 P 70d7b8f0a9d34ae48a61a19cfe8cb141 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 70d7b8f0a9d34ae48a61a19cfe8cb141, State: Initialized, Role: FOLLOWER
I20260809 09:37:03.918394 20186 consensus_queue.cc:260] T 00000000000000000000000000000000 P 70d7b8f0a9d34ae48a61a19cfe8cb141 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70d7b8f0a9d34ae48a61a19cfe8cb141" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35749 } }
I20260809 09:37:03.918593 20186 raft_consensus.cc:388] T 00000000000000000000000000000000 P 70d7b8f0a9d34ae48a61a19cfe8cb141 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 09:37:03.918677 20186 raft_consensus.cc:482] T 00000000000000000000000000000000 P 70d7b8f0a9d34ae48a61a19cfe8cb141 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 09:37:03.918805 20186 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 70d7b8f0a9d34ae48a61a19cfe8cb141 [term 0 FOLLOWER]: Advancing to term 1
I20260809 09:37:03.919852 20186 raft_consensus.cc:504] T 00000000000000000000000000000000 P 70d7b8f0a9d34ae48a61a19cfe8cb141 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70d7b8f0a9d34ae48a61a19cfe8cb141" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35749 } }
I20260809 09:37:03.920244 20186 leader_election.cc:302] T 00000000000000000000000000000000 P 70d7b8f0a9d34ae48a61a19cfe8cb141 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 70d7b8f0a9d34ae48a61a19cfe8cb141; no voters:
I20260809 09:37:03.920608 20186 leader_election.cc:288] T 00000000000000000000000000000000 P 70d7b8f0a9d34ae48a61a19cfe8cb141 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 09:37:03.920778 20192 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 70d7b8f0a9d34ae48a61a19cfe8cb141 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 09:37:03.920995 20192 raft_consensus.cc:686] T 00000000000000000000000000000000 P 70d7b8f0a9d34ae48a61a19cfe8cb141 [term 1 LEADER]: Becoming Leader. State: Replica: 70d7b8f0a9d34ae48a61a19cfe8cb141, State: Running, Role: LEADER
I20260809 09:37:03.921324 20192 consensus_queue.cc:237] T 00000000000000000000000000000000 P 70d7b8f0a9d34ae48a61a19cfe8cb141 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70d7b8f0a9d34ae48a61a19cfe8cb141" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35749 } }
I20260809 09:37:03.921708 20189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:37:03.906445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.136:39847 (local address 127.0.0.1:35749)
0809 09:37:03.906930 (+ 485us) server_negotiation.cc:206] Beginning negotiation
0809 09:37:03.906947 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0809 09:37:03.906982 (+ 35us) server_negotiation.cc:407] Connection header received
0809 09:37:03.907639 (+ 657us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:37:03.907656 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:37:03.907992 (+ 336us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:37:03.908270 (+ 278us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:37:03.909318 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:37:03.910354 (+ 1036us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:37:03.911676 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:37:03.912172 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:37:03.912366 (+ 194us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:37:03.915442 (+ 3076us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:37:03.915448 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:37:03.915461 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:37:03.915490 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:37:03.918145 (+ 2655us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:37:03.918784 (+ 639us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:37:03.918793 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:37:03.918799 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:37:03.918970 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:37:03.919406 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:37:03.919409 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:37:03.919410 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:37:03.919805 (+ 395us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:37:03.920017 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0809 09:37:03.920335 (+ 318us) server_negotiation.cc:299] Negotiation successful
0809 09:37:03.920631 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":290,"thread_start_us":140,"threads_started":1}
I20260809 09:37:03.921958 20186 sys_catalog.cc:564] T 00000000000000000000000000000000 P 70d7b8f0a9d34ae48a61a19cfe8cb141 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 09:37:03.922549 20192 sys_catalog.cc:455] T 00000000000000000000000000000000 P 70d7b8f0a9d34ae48a61a19cfe8cb141 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 70d7b8f0a9d34ae48a61a19cfe8cb141. Latest consensus state: current_term: 1 leader_uuid: "70d7b8f0a9d34ae48a61a19cfe8cb141" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70d7b8f0a9d34ae48a61a19cfe8cb141" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35749 } } }
I20260809 09:37:03.922670 20192 sys_catalog.cc:458] T 00000000000000000000000000000000 P 70d7b8f0a9d34ae48a61a19cfe8cb141 [sys.catalog]: This master's current role is: LEADER
I20260809 09:37:03.923022 20199 sys_catalog.cc:455] T 00000000000000000000000000000000 P 70d7b8f0a9d34ae48a61a19cfe8cb141 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "70d7b8f0a9d34ae48a61a19cfe8cb141" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70d7b8f0a9d34ae48a61a19cfe8cb141" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35749 } } }
I20260809 09:37:03.923040 20203 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 09:37:03.923134 20199 sys_catalog.cc:458] T 00000000000000000000000000000000 P 70d7b8f0a9d34ae48a61a19cfe8cb141 [sys.catalog]: This master's current role is: LEADER
I20260809 09:37:03.925004 20982 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 09:37:03.926272 20203 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 09:37:03.932578 20203 catalog_manager.cc:1347] Generated new cluster ID: 9cc9d49344f047b4bfa4f3f452fa221a
I20260809 09:37:03.932695 20203 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 09:37:03.936522 20189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:37:03.927201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.136:45517 (local address 127.0.0.1:35749)
0809 09:37:03.927335 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0809 09:37:03.927339 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 09:37:03.927735 (+ 396us) server_negotiation.cc:407] Connection header received
0809 09:37:03.927868 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:37:03.927871 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:37:03.927915 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:37:03.928010 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:37:03.928749 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:37:03.929507 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:37:03.930642 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:37:03.930933 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:37:03.931044 (+ 111us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:37:03.932774 (+ 1730us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:37:03.932778 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:37:03.932781 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:37:03.932807 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:37:03.934614 (+ 1807us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:37:03.935154 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:37:03.935157 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:37:03.935159 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:37:03.935253 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:37:03.935589 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:37:03.935593 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:37:03.935595 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:37:03.935817 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:37:03.935983 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0809 09:37:03.936242 (+ 259us) server_negotiation.cc:299] Negotiation successful
0809 09:37:03.936344 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260809 09:37:03.956954 20203 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 09:37:03.957870 20203 catalog_manager.cc:1504] Loading token signing keys...
I20260809 09:37:03.966306 20203 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 70d7b8f0a9d34ae48a61a19cfe8cb141: Generated new TSK 0
I20260809 09:37:03.967111 20203 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 09:37:03.973507 20982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu with pid 20076
W20260809 09:37:03.983609 20050 connection.cc:535] client connection to 127.0.0.1:35749 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:35749): Connection reset by peer (error 104)
2026-08-09T09:37:03Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1636 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786267941944493-20982-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 09:37:04 dist-test-slave-7kzw krb5kdc[20239](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 09:37:04 dist-test-slave-7kzw krb5kdc[20239](info): set up 2 sockets
Aug 09 09:37:04 dist-test-slave-7kzw krb5kdc[20239](info): commencing operation
krb5kdc: starting...
W20260809 09:37:05.221236 20982 mini_kdc.cc:121] Time spent starting KDC: real 1.222s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786267941944493-20982-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786267941944493-20982-0/krb5kdc/test-user.keytab.
Aug 09 09:37:05 dist-test-slave-7kzw krb5kdc[20239](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268225, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T09:37:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T09:37:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786267941944493-20982-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786267941944493-20982-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 09:37:05.370647 20982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
/tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786267941944493-20982-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786267941944493-20982-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786267941944493-20982-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786267941944493-20982-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43463
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786267941944493-20982-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34559
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43463
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786267941944493-20982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786267941944493-20982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786267941944493-20982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 09:37:05.486413 20259 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 09:37:05.486697 20259 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 09:37:05.486739 20259 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 09:37:05.490897 20259 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 09:37:05.490967 20259 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 09:37:05.490986 20259 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 09:37:05.491003 20259 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 09:37:05.491019 20259 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 09:37:05.496429 20259 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34559
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786267941944493-20982-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786267941944493-20982-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43463
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786267941944493-20982-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43463
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786267941944493-20982-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20259
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786267941944493-20982-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 08:38:55 UTC on ccycloud.quasar-njjleu.root.comops.site
build id 4575935
I20260809 09:37:05.497615 20259 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 09:37:05.498792 20259 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 09:37:05.508405 20268 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:37:05.508411 20265 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:37:05.508600 20259 server_base.cc:1034] running on GCE node
W20260809 09:37:05.508405 20264 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:37:05.509429 20259 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 09:37:05.510672 20259 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 09:37:05.511857 20259 hybrid_clock.cc:648] HybridClock initialized: now 1786268225511825 us; error 43 us; skew 500 ppm
Aug 09 09:37:05 dist-test-slave-7kzw krb5kdc[20239](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268225, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 09:37:05.514796 20259 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 09:37:05.515771 20259 webserver.cc:460] Webserver started at http://127.0.0.1:38895/ using document root <none> and password file <none>
I20260809 09:37:05.516353 20259 fs_manager.cc:359] Metadata directory not provided
I20260809 09:37:05.516418 20259 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 09:37:05.516681 20259 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 09:37:05.518894 20259 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786267941944493-20982-0/minicluster-data/master-0/data/instance:
uuid: "3842eed0199f4c36acce5b29a05ce009"
format_stamp: "Formatted at 2026-08-09 09:37:05 on dist-test-slave-7kzw"
server_key: "f6676cf15d43e291f513e8369ff16c86"
server_key_iv: "d260410a2f0749d82f4569d734b8d96d"
server_key_version: "encryptionkey@0"
I20260809 09:37:05.519479 20259 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786267941944493-20982-0/minicluster-data/master-0/wal/instance:
uuid: "3842eed0199f4c36acce5b29a05ce009"
format_stamp: "Formatted at 2026-08-09 09:37:05 on dist-test-slave-7kzw"
server_key: "f6676cf15d43e291f513e8369ff16c86"
server_key_iv: "d260410a2f0749d82f4569d734b8d96d"
server_key_version: "encryptionkey@0"
I20260809 09:37:05.523269 20259 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260809 09:37:05.525703 20277 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 09:37:05.526925 20259 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260809 09:37:05.527083 20259 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786267941944493-20982-0/minicluster-data/master-0/data,/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786267941944493-20982-0/minicluster-data/master-0/wal
uuid: "3842eed0199f4c36acce5b29a05ce009"
format_stamp: "Formatted at 2026-08-09 09:37:05 on dist-test-slave-7kzw"
server_key: "f6676cf15d43e291f513e8369ff16c86"
server_key_iv: "d260410a2f0749d82f4569d734b8d96d"
server_key_version: "encryptionkey@0"
I20260809 09:37:05.527199 20259 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786267941944493-20982-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786267941944493-20982-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786267941944493-20982-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 09:37:05.541491 20259 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 09:37:05.544615 20259 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 09:37:05.544844 20259 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 09:37:05.552650 20259 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43463
I20260809 09:37:05.552676 20353 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43463 every 8 connection(s)
I20260809 09:37:05.553642 20259 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786267941944493-20982-0/minicluster-data/master-0/data/info.pb
I20260809 09:37:05.556757 20357 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 09:37:05.558805 20982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu as pid 20259
I20260809 09:37:05.558974 20982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786267941944493-20982-0/minicluster-data/master-0/wal/instance
I20260809 09:37:05.559270 20982 external_mini_cluster.cc:1428] Setting key dc4d46db7769c8bbdf39c21cb5db46ac
I20260809 09:37:05.564110 20357 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3842eed0199f4c36acce5b29a05ce009: Bootstrap starting.
I20260809 09:37:05.567289 20357 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3842eed0199f4c36acce5b29a05ce009: Neither blocks nor log segments found. Creating new log.
I20260809 09:37:05.568245 20357 log.cc:824] T 00000000000000000000000000000000 P 3842eed0199f4c36acce5b29a05ce009: Log is configured to *not* fsync() on all Append() calls
I20260809 09:37:05.569404 20363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:37:05.560839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.136:59251 (local address 127.0.0.1:43463)
0809 09:37:05.561293 (+ 454us) server_negotiation.cc:206] Beginning negotiation
0809 09:37:05.561310 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0809 09:37:05.561347 (+ 37us) server_negotiation.cc:407] Connection header received
0809 09:37:05.562009 (+ 662us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:37:05.562027 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:37:05.562370 (+ 343us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:37:05.562732 (+ 362us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:37:05.563196 (+ 464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:37:05.564187 (+ 991us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:37:05.565547 (+ 1360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:37:05.566031 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:37:05.566172 (+ 141us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:37:05.567283 (+ 1111us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:37:05.567288 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:37:05.567301 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 09:37:05.567785 (+ 484us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:37:05.567844 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 09:37:05.567868 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:37:05.567974 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0809 09:37:05.568310 (+ 336us) server_negotiation.cc:299] Negotiation successful
0809 09:37:05.568462 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":290,"thread_start_us":143,"threads_started":1}
I20260809 09:37:05.570792 20357 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3842eed0199f4c36acce5b29a05ce009: No bootstrap required, opened a new log
I20260809 09:37:05.572642 20982 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 09:37:05.575758 20357 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3842eed0199f4c36acce5b29a05ce009 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3842eed0199f4c36acce5b29a05ce009" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43463 } }
I20260809 09:37:05.576110 20357 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3842eed0199f4c36acce5b29a05ce009 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 09:37:05.576197 20357 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3842eed0199f4c36acce5b29a05ce009 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3842eed0199f4c36acce5b29a05ce009, State: Initialized, Role: FOLLOWER
I20260809 09:37:05.576735 20357 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3842eed0199f4c36acce5b29a05ce009 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3842eed0199f4c36acce5b29a05ce009" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43463 } }
I20260809 09:37:05.576871 20357 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3842eed0199f4c36acce5b29a05ce009 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 09:37:05.576975 20357 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3842eed0199f4c36acce5b29a05ce009 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 09:37:05.577102 20357 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3842eed0199f4c36acce5b29a05ce009 [term 0 FOLLOWER]: Advancing to term 1
I20260809 09:37:05.578131 20357 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3842eed0199f4c36acce5b29a05ce009 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3842eed0199f4c36acce5b29a05ce009" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43463 } }
I20260809 09:37:05.578572 20357 leader_election.cc:302] T 00000000000000000000000000000000 P 3842eed0199f4c36acce5b29a05ce009 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3842eed0199f4c36acce5b29a05ce009; no voters:
I20260809 09:37:05.579025 20357 leader_election.cc:288] T 00000000000000000000000000000000 P 3842eed0199f4c36acce5b29a05ce009 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 09:37:05.579103 20370 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3842eed0199f4c36acce5b29a05ce009 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 09:37:05.579391 20370 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3842eed0199f4c36acce5b29a05ce009 [term 1 LEADER]: Becoming Leader. State: Replica: 3842eed0199f4c36acce5b29a05ce009, State: Running, Role: LEADER
I20260809 09:37:05.579747 20363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:37:05.575385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.136:42269 (local address 127.0.0.1:43463)
0809 09:37:05.575506 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0809 09:37:05.575510 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 09:37:05.575731 (+ 221us) server_negotiation.cc:407] Connection header received
0809 09:37:05.575868 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:37:05.575871 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:37:05.575912 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:37:05.575979 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:37:05.576389 (+ 410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:37:05.576941 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:37:05.577868 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:37:05.578116 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:37:05.578247 (+ 131us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:37:05.579211 (+ 964us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:37:05.579216 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:37:05.579219 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 09:37:05.579234 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:37:05.579264 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 09:37:05.579281 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:37:05.579380 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0809 09:37:05.579560 (+ 180us) server_negotiation.cc:299] Negotiation successful
0809 09:37:05.579594 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260809 09:37:05.579768 20370 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3842eed0199f4c36acce5b29a05ce009 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3842eed0199f4c36acce5b29a05ce009" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43463 } }
I20260809 09:37:05.580565 20357 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3842eed0199f4c36acce5b29a05ce009 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 09:37:05.581864 20371 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3842eed0199f4c36acce5b29a05ce009 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3842eed0199f4c36acce5b29a05ce009. Latest consensus state: current_term: 1 leader_uuid: "3842eed0199f4c36acce5b29a05ce009" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3842eed0199f4c36acce5b29a05ce009" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43463 } } }
I20260809 09:37:05.581874 20373 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3842eed0199f4c36acce5b29a05ce009 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3842eed0199f4c36acce5b29a05ce009" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3842eed0199f4c36acce5b29a05ce009" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43463 } } }
I20260809 09:37:05.582001 20371 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3842eed0199f4c36acce5b29a05ce009 [sys.catalog]: This master's current role is: LEADER
I20260809 09:37:05.582013 20373 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3842eed0199f4c36acce5b29a05ce009 [sys.catalog]: This master's current role is: LEADER
I20260809 09:37:05.582494 20385 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 09:37:05.585858 20385 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 09:37:05.592021 20385 catalog_manager.cc:1347] Generated new cluster ID: 28e49c72a425400b8e61b08afb53d06b
I20260809 09:37:05.592123 20385 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 09:37:05.602497 20385 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 09:37:05.603461 20385 catalog_manager.cc:1504] Loading token signing keys...
I20260809 09:37:05.613678 20385 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3842eed0199f4c36acce5b29a05ce009: Generated new TSK 0
I20260809 09:37:05.614388 20385 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 09:37:05.619596 20982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu with pid 20259
W20260809 09:37:05.630312 20236 connection.cc:535] client connection to 127.0.0.1:43463 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:43463): Connection reset by peer (error 104)
2026-08-09T09:37:05Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1646 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786267941944493-20982-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 09:37:05 dist-test-slave-7kzw krb5kdc[20400](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 09:37:05 dist-test-slave-7kzw krb5kdc[20400](info): set up 2 sockets
Aug 09 09:37:05 dist-test-slave-7kzw krb5kdc[20400](info): commencing operation
krb5kdc: starting...
W20260809 09:37:06.863044 20982 mini_kdc.cc:121] Time spent starting KDC: real 1.218s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786267941944493-20982-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786267941944493-20982-0/krb5kdc/test-user.keytab.
Aug 09 09:37:06 dist-test-slave-7kzw krb5kdc[20400](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268226, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T09:37:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T09:37:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786267941944493-20982-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786267941944493-20982-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 09:37:07.007171 20982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
/tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786267941944493-20982-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786267941944493-20982-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786267941944493-20982-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786267941944493-20982-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39051
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786267941944493-20982-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44895
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39051
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786267941944493-20982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786267941944493-20982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786267941944493-20982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 09:37:07.127487 20417 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 09:37:07.127784 20417 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 09:37:07.127827 20417 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 09:37:07.131910 20417 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 09:37:07.131975 20417 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 09:37:07.131995 20417 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 09:37:07.132012 20417 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 09:37:07.132027 20417 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 09:37:07.137440 20417 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44895
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786267941944493-20982-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786267941944493-20982-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39051
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786267941944493-20982-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39051
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786267941944493-20982-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20417
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786267941944493-20982-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 08:38:55 UTC on ccycloud.quasar-njjleu.root.comops.site
build id 4575935
I20260809 09:37:07.138546 20417 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 09:37:07.139593 20417 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 09:37:07.148156 20428 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:37:07.148147 20425 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:37:07.148407 20417 server_base.cc:1034] running on GCE node
W20260809 09:37:07.148244 20431 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:37:07.149161 20417 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 09:37:07.150306 20417 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 09:37:07.151510 20417 hybrid_clock.cc:648] HybridClock initialized: now 1786268227151494 us; error 25 us; skew 500 ppm
Aug 09 09:37:07 dist-test-slave-7kzw krb5kdc[20400](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268227, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 09:37:07.154526 20417 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 09:37:07.155416 20417 webserver.cc:460] Webserver started at http://127.0.0.1:46175/ using document root <none> and password file <none>
I20260809 09:37:07.155989 20417 fs_manager.cc:359] Metadata directory not provided
I20260809 09:37:07.156087 20417 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 09:37:07.156329 20417 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 09:37:07.158421 20417 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786267941944493-20982-0/minicluster-data/master-0/data/instance:
uuid: "946fb1792b3a4074a6ab7a8324e5cbed"
format_stamp: "Formatted at 2026-08-09 09:37:07 on dist-test-slave-7kzw"
server_key: "efbff64d8596591eb79427ddedd9ab31"
server_key_iv: "f357c4d579ae1c96e49b51b5835551b0"
server_key_version: "encryptionkey@0"
I20260809 09:37:07.159075 20417 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786267941944493-20982-0/minicluster-data/master-0/wal/instance:
uuid: "946fb1792b3a4074a6ab7a8324e5cbed"
format_stamp: "Formatted at 2026-08-09 09:37:07 on dist-test-slave-7kzw"
server_key: "efbff64d8596591eb79427ddedd9ab31"
server_key_iv: "f357c4d579ae1c96e49b51b5835551b0"
server_key_version: "encryptionkey@0"
I20260809 09:37:07.163057 20417 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260809 09:37:07.165740 20442 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 09:37:07.167215 20417 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 09:37:07.167377 20417 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786267941944493-20982-0/minicluster-data/master-0/data,/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786267941944493-20982-0/minicluster-data/master-0/wal
uuid: "946fb1792b3a4074a6ab7a8324e5cbed"
format_stamp: "Formatted at 2026-08-09 09:37:07 on dist-test-slave-7kzw"
server_key: "efbff64d8596591eb79427ddedd9ab31"
server_key_iv: "f357c4d579ae1c96e49b51b5835551b0"
server_key_version: "encryptionkey@0"
I20260809 09:37:07.167502 20417 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786267941944493-20982-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786267941944493-20982-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786267941944493-20982-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 09:37:07.172295 20417 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 09:37:07.175863 20417 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 09:37:07.176069 20417 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 09:37:07.183894 20417 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39051
I20260809 09:37:07.183987 20516 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39051 every 8 connection(s)
I20260809 09:37:07.184955 20417 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786267941944493-20982-0/minicluster-data/master-0/data/info.pb
I20260809 09:37:07.185271 20982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu as pid 20417
I20260809 09:37:07.185462 20982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786267941944493-20982-0/minicluster-data/master-0/wal/instance
I20260809 09:37:07.185913 20982 external_mini_cluster.cc:1428] Setting key c595dc67afbc73349dbe0df7c7f3811b
I20260809 09:37:07.188611 20518 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 09:37:07.191808 20522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:37:07.187419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.136:34041 (local address 127.0.0.1:39051)
0809 09:37:07.187870 (+ 451us) server_negotiation.cc:206] Beginning negotiation
0809 09:37:07.187890 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0809 09:37:07.187935 (+ 45us) server_negotiation.cc:407] Connection header received
0809 09:37:07.188804 (+ 869us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:37:07.188828 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:37:07.189190 (+ 362us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:37:07.189541 (+ 351us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:37:07.190364 (+ 823us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:37:07.190370 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:37:07.190386 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 09:37:07.190399 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:37:07.190460 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 09:37:07.190514 (+ 54us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:37:07.190658 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0809 09:37:07.190755 (+ 97us) server_negotiation.cc:299] Negotiation successful
0809 09:37:07.190911 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":274,"thread_start_us":115,"threads_started":1}
I20260809 09:37:07.194844 20982 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 09:37:07.195458 20518 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 946fb1792b3a4074a6ab7a8324e5cbed: Bootstrap starting.
I20260809 09:37:07.197909 20518 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 946fb1792b3a4074a6ab7a8324e5cbed: Neither blocks nor log segments found. Creating new log.
I20260809 09:37:07.198810 20518 log.cc:824] T 00000000000000000000000000000000 P 946fb1792b3a4074a6ab7a8324e5cbed: Log is configured to *not* fsync() on all Append() calls
I20260809 09:37:07.198939 20522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:37:07.197074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.136:57607 (local address 127.0.0.1:39051)
0809 09:37:07.197197 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0809 09:37:07.197201 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 09:37:07.197403 (+ 202us) server_negotiation.cc:407] Connection header received
0809 09:37:07.197501 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:37:07.197505 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:37:07.197538 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:37:07.197613 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:37:07.198444 (+ 831us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:37:07.198447 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:37:07.198450 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 09:37:07.198501 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:37:07.198532 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 09:37:07.198551 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:37:07.198647 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0809 09:37:07.198773 (+ 126us) server_negotiation.cc:299] Negotiation successful
0809 09:37:07.198814 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260809 09:37:07.200790 20518 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 946fb1792b3a4074a6ab7a8324e5cbed: No bootstrap required, opened a new log
I20260809 09:37:07.204213 20518 raft_consensus.cc:348] T 00000000000000000000000000000000 P 946fb1792b3a4074a6ab7a8324e5cbed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "946fb1792b3a4074a6ab7a8324e5cbed" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39051 } }
I20260809 09:37:07.204443 20518 raft_consensus.cc:374] T 00000000000000000000000000000000 P 946fb1792b3a4074a6ab7a8324e5cbed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 09:37:07.204507 20518 raft_consensus.cc:729] T 00000000000000000000000000000000 P 946fb1792b3a4074a6ab7a8324e5cbed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 946fb1792b3a4074a6ab7a8324e5cbed, State: Initialized, Role: FOLLOWER
I20260809 09:37:07.204983 20518 consensus_queue.cc:260] T 00000000000000000000000000000000 P 946fb1792b3a4074a6ab7a8324e5cbed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "946fb1792b3a4074a6ab7a8324e5cbed" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39051 } }
I20260809 09:37:07.205114 20518 raft_consensus.cc:388] T 00000000000000000000000000000000 P 946fb1792b3a4074a6ab7a8324e5cbed [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 09:37:07.205202 20518 raft_consensus.cc:482] T 00000000000000000000000000000000 P 946fb1792b3a4074a6ab7a8324e5cbed [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 09:37:07.205295 20518 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 946fb1792b3a4074a6ab7a8324e5cbed [term 0 FOLLOWER]: Advancing to term 1
I20260809 09:37:07.206236 20518 raft_consensus.cc:504] T 00000000000000000000000000000000 P 946fb1792b3a4074a6ab7a8324e5cbed [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "946fb1792b3a4074a6ab7a8324e5cbed" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39051 } }
I20260809 09:37:07.206645 20518 leader_election.cc:302] T 00000000000000000000000000000000 P 946fb1792b3a4074a6ab7a8324e5cbed [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 946fb1792b3a4074a6ab7a8324e5cbed; no voters:
I20260809 09:37:07.206970 20518 leader_election.cc:288] T 00000000000000000000000000000000 P 946fb1792b3a4074a6ab7a8324e5cbed [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 09:37:07.207165 20532 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 946fb1792b3a4074a6ab7a8324e5cbed [term 1 FOLLOWER]: Leader election won for term 1
I20260809 09:37:07.207607 20532 raft_consensus.cc:686] T 00000000000000000000000000000000 P 946fb1792b3a4074a6ab7a8324e5cbed [term 1 LEADER]: Becoming Leader. State: Replica: 946fb1792b3a4074a6ab7a8324e5cbed, State: Running, Role: LEADER
I20260809 09:37:07.207998 20532 consensus_queue.cc:237] T 00000000000000000000000000000000 P 946fb1792b3a4074a6ab7a8324e5cbed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "946fb1792b3a4074a6ab7a8324e5cbed" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39051 } }
I20260809 09:37:07.208319 20518 sys_catalog.cc:564] T 00000000000000000000000000000000 P 946fb1792b3a4074a6ab7a8324e5cbed [sys.catalog]: configured and running, proceeding with master startup.
I20260809 09:37:07.210009 20533 sys_catalog.cc:455] T 00000000000000000000000000000000 P 946fb1792b3a4074a6ab7a8324e5cbed [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "946fb1792b3a4074a6ab7a8324e5cbed" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "946fb1792b3a4074a6ab7a8324e5cbed" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39051 } } }
I20260809 09:37:07.210078 20534 sys_catalog.cc:455] T 00000000000000000000000000000000 P 946fb1792b3a4074a6ab7a8324e5cbed [sys.catalog]: SysCatalogTable state changed. Reason: New leader 946fb1792b3a4074a6ab7a8324e5cbed. Latest consensus state: current_term: 1 leader_uuid: "946fb1792b3a4074a6ab7a8324e5cbed" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "946fb1792b3a4074a6ab7a8324e5cbed" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39051 } } }
I20260809 09:37:07.210175 20533 sys_catalog.cc:458] T 00000000000000000000000000000000 P 946fb1792b3a4074a6ab7a8324e5cbed [sys.catalog]: This master's current role is: LEADER
I20260809 09:37:07.210199 20534 sys_catalog.cc:458] T 00000000000000000000000000000000 P 946fb1792b3a4074a6ab7a8324e5cbed [sys.catalog]: This master's current role is: LEADER
I20260809 09:37:07.210631 20549 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 09:37:07.213769 20549 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 09:37:07.219835 20549 catalog_manager.cc:1347] Generated new cluster ID: 60445bfa36ef46b8a33572c9bdc5ca59
I20260809 09:37:07.219916 20549 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 09:37:07.255033 20549 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 09:37:07.255461 20549 catalog_manager.cc:1504] Loading token signing keys...
I20260809 09:37:07.271046 20549 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 946fb1792b3a4074a6ab7a8324e5cbed: Generated new TSK 0
I20260809 09:37:07.272006 20549 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 09:37:07.279326 20982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu with pid 20417
2026-08-09T09:37:07Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1661 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786267941944493-20982-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 09:37:07 dist-test-slave-7kzw krb5kdc[20557](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 09:37:07 dist-test-slave-7kzw krb5kdc[20557](info): set up 2 sockets
Aug 09 09:37:07 dist-test-slave-7kzw krb5kdc[20557](info): commencing operation
krb5kdc: starting...
W20260809 09:37:08.559719 20982 mini_kdc.cc:121] Time spent starting KDC: real 1.253s user 0.008s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786267941944493-20982-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786267941944493-20982-0/krb5kdc/test-user.keytab.
Aug 09 09:37:08 dist-test-slave-7kzw krb5kdc[20557](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268228, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T09:37:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T09:37:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.125.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.125.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.125.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.190.keytab.
Entry for principal kudu/127.20.125.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.190.keytab.
I20260809 09:37:08.701611 20982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
/tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786267941944493-20982-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786267941944493-20982-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786267941944493-20982-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786267941944493-20982-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.125.190:43101
--webserver_interface=127.20.125.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.190.keytab
--principal=kudu/127.20.125.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.125.148:33943
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.125.190:43101
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786267941944493-20982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786267941944493-20982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786267941944493-20982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 09:37:08.816608 20573 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 09:37:08.816934 20573 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 09:37:08.817018 20573 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 09:37:08.821165 20573 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 09:37:08.821259 20573 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 09:37:08.821300 20573 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 09:37:08.821337 20573 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 09:37:08.821372 20573 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 09:37:08.827076 20573 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.125.148:33943
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786267941944493-20982-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786267941944493-20982-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.125.190:43101
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.190.keytab
--principal=kudu/127.20.125.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.125.190:43101
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786267941944493-20982-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.125.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20573
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786267941944493-20982-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 08:38:55 UTC on ccycloud.quasar-njjleu.root.comops.site
build id 4575935
I20260809 09:37:08.828197 20573 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 09:37:08.829257 20573 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 09:37:08.838346 20586 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:37:08.838347 20581 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:37:08.838347 20583 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:37:08.838445 20573 server_base.cc:1034] running on GCE node
I20260809 09:37:08.839351 20573 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 09:37:08.840492 20573 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 09:37:08.841758 20573 hybrid_clock.cc:648] HybridClock initialized: now 1786268228841729 us; error 46 us; skew 500 ppm
Aug 09 09:37:08 dist-test-slave-7kzw krb5kdc[20557](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268228, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 09:37:08.845572 20573 init.cc:377] Logged in from keytab as kudu/127.20.125.190@KRBTEST.COM (short username kudu)
I20260809 09:37:08.847008 20573 webserver.cc:460] Webserver started at http://127.20.125.190:36131/ using document root <none> and password file <none>
I20260809 09:37:08.847898 20573 fs_manager.cc:359] Metadata directory not provided
I20260809 09:37:08.848066 20573 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 09:37:08.848435 20573 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 09:37:08.852156 20573 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786267941944493-20982-0/minicluster-data/master-0/data/instance:
uuid: "b84d03dbf5564fe3bdef8d6d8311ab0a"
format_stamp: "Formatted at 2026-08-09 09:37:08 on dist-test-slave-7kzw"
server_key: "0c21ffb40c9967454e678e19db27ab50"
server_key_iv: "abc6350768bc65cd4283ab6ebfd037b1"
server_key_version: "encryptionkey@0"
I20260809 09:37:08.853073 20573 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786267941944493-20982-0/minicluster-data/master-0/wal/instance:
uuid: "b84d03dbf5564fe3bdef8d6d8311ab0a"
format_stamp: "Formatted at 2026-08-09 09:37:08 on dist-test-slave-7kzw"
server_key: "0c21ffb40c9967454e678e19db27ab50"
server_key_iv: "abc6350768bc65cd4283ab6ebfd037b1"
server_key_version: "encryptionkey@0"
I20260809 09:37:08.858346 20573 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 09:37:08.861019 20603 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 09:37:08.862426 20573 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 09:37:08.862643 20573 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786267941944493-20982-0/minicluster-data/master-0/data,/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786267941944493-20982-0/minicluster-data/master-0/wal
uuid: "b84d03dbf5564fe3bdef8d6d8311ab0a"
format_stamp: "Formatted at 2026-08-09 09:37:08 on dist-test-slave-7kzw"
server_key: "0c21ffb40c9967454e678e19db27ab50"
server_key_iv: "abc6350768bc65cd4283ab6ebfd037b1"
server_key_version: "encryptionkey@0"
I20260809 09:37:08.862776 20573 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786267941944493-20982-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786267941944493-20982-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786267941944493-20982-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 09:37:08.898139 20573 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 09:37:08.902153 20573 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 09:37:08.902395 20573 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 09:37:08.910219 20573 rpc_server.cc:307] RPC server started. Bound to: 127.20.125.190:43101
I20260809 09:37:08.910290 20681 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.125.190:43101 every 8 connection(s)
I20260809 09:37:08.911408 20573 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786267941944493-20982-0/minicluster-data/master-0/data/info.pb
I20260809 09:37:08.914431 20682 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 09:37:08.920097 20982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu as pid 20573
I20260809 09:37:08.920145 20682 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b84d03dbf5564fe3bdef8d6d8311ab0a: Bootstrap starting.
I20260809 09:37:08.920285 20982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786267941944493-20982-0/minicluster-data/master-0/wal/instance
I20260809 09:37:08.920567 20982 external_mini_cluster.cc:1428] Setting key 260bd59e26b34d6f644da433f10d817a
I20260809 09:37:08.922863 20682 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b84d03dbf5564fe3bdef8d6d8311ab0a: Neither blocks nor log segments found. Creating new log.
I20260809 09:37:08.923617 20682 log.cc:824] T 00000000000000000000000000000000 P b84d03dbf5564fe3bdef8d6d8311ab0a: Log is configured to *not* fsync() on all Append() calls
I20260809 09:37:08.925540 20682 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b84d03dbf5564fe3bdef8d6d8311ab0a: No bootstrap required, opened a new log
I20260809 09:37:08.929075 20682 raft_consensus.cc:348] T 00000000000000000000000000000000 P b84d03dbf5564fe3bdef8d6d8311ab0a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b84d03dbf5564fe3bdef8d6d8311ab0a" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 43101 } }
I20260809 09:37:08.929333 20682 raft_consensus.cc:374] T 00000000000000000000000000000000 P b84d03dbf5564fe3bdef8d6d8311ab0a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 09:37:08.929404 20682 raft_consensus.cc:729] T 00000000000000000000000000000000 P b84d03dbf5564fe3bdef8d6d8311ab0a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b84d03dbf5564fe3bdef8d6d8311ab0a, State: Initialized, Role: FOLLOWER
Aug 09 09:37:08 dist-test-slave-7kzw krb5kdc[20557](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268228, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.125.190@KRBTEST.COM
I20260809 09:37:08.929960 20682 consensus_queue.cc:260] T 00000000000000000000000000000000 P b84d03dbf5564fe3bdef8d6d8311ab0a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b84d03dbf5564fe3bdef8d6d8311ab0a" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 43101 } }
I20260809 09:37:08.930109 20682 raft_consensus.cc:388] T 00000000000000000000000000000000 P b84d03dbf5564fe3bdef8d6d8311ab0a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 09:37:08.930194 20682 raft_consensus.cc:482] T 00000000000000000000000000000000 P b84d03dbf5564fe3bdef8d6d8311ab0a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 09:37:08.930326 20682 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b84d03dbf5564fe3bdef8d6d8311ab0a [term 0 FOLLOWER]: Advancing to term 1
I20260809 09:37:08.931456 20682 raft_consensus.cc:504] T 00000000000000000000000000000000 P b84d03dbf5564fe3bdef8d6d8311ab0a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b84d03dbf5564fe3bdef8d6d8311ab0a" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 43101 } }
I20260809 09:37:08.931849 20682 leader_election.cc:302] T 00000000000000000000000000000000 P b84d03dbf5564fe3bdef8d6d8311ab0a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b84d03dbf5564fe3bdef8d6d8311ab0a; no voters:
I20260809 09:37:08.932214 20682 leader_election.cc:288] T 00000000000000000000000000000000 P b84d03dbf5564fe3bdef8d6d8311ab0a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 09:37:08.932296 20695 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b84d03dbf5564fe3bdef8d6d8311ab0a [term 1 FOLLOWER]: Leader election won for term 1
I20260809 09:37:08.932516 20695 raft_consensus.cc:686] T 00000000000000000000000000000000 P b84d03dbf5564fe3bdef8d6d8311ab0a [term 1 LEADER]: Becoming Leader. State: Replica: b84d03dbf5564fe3bdef8d6d8311ab0a, State: Running, Role: LEADER
I20260809 09:37:08.932879 20695 consensus_queue.cc:237] T 00000000000000000000000000000000 P b84d03dbf5564fe3bdef8d6d8311ab0a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b84d03dbf5564fe3bdef8d6d8311ab0a" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 43101 } }
I20260809 09:37:08.933554 20682 sys_catalog.cc:564] T 00000000000000000000000000000000 P b84d03dbf5564fe3bdef8d6d8311ab0a [sys.catalog]: configured and running, proceeding with master startup.
I20260809 09:37:08.934202 20695 sys_catalog.cc:455] T 00000000000000000000000000000000 P b84d03dbf5564fe3bdef8d6d8311ab0a [sys.catalog]: SysCatalogTable state changed. Reason: New leader b84d03dbf5564fe3bdef8d6d8311ab0a. Latest consensus state: current_term: 1 leader_uuid: "b84d03dbf5564fe3bdef8d6d8311ab0a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b84d03dbf5564fe3bdef8d6d8311ab0a" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 43101 } } }
I20260809 09:37:08.934405 20695 sys_catalog.cc:458] T 00000000000000000000000000000000 P b84d03dbf5564fe3bdef8d6d8311ab0a [sys.catalog]: This master's current role is: LEADER
I20260809 09:37:08.934659 20696 sys_catalog.cc:455] T 00000000000000000000000000000000 P b84d03dbf5564fe3bdef8d6d8311ab0a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b84d03dbf5564fe3bdef8d6d8311ab0a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b84d03dbf5564fe3bdef8d6d8311ab0a" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 43101 } } }
I20260809 09:37:08.934830 20696 sys_catalog.cc:458] T 00000000000000000000000000000000 P b84d03dbf5564fe3bdef8d6d8311ab0a [sys.catalog]: This master's current role is: LEADER
I20260809 09:37:08.934952 20705 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 09:37:08.937115 20689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:37:08.921939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39081 (local address 127.20.125.190:43101)
0809 09:37:08.922447 (+ 508us) server_negotiation.cc:206] Beginning negotiation
0809 09:37:08.922517 (+ 70us) server_negotiation.cc:399] Waiting for connection header
0809 09:37:08.922582 (+ 65us) server_negotiation.cc:407] Connection header received
0809 09:37:08.923333 (+ 751us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:37:08.923354 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:37:08.923714 (+ 360us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:37:08.924107 (+ 393us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:37:08.925017 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:37:08.926123 (+ 1106us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:37:08.927090 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:37:08.927585 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:37:08.927776 (+ 191us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:37:08.930645 (+ 2869us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:37:08.930650 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:37:08.930664 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:37:08.930693 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:37:08.932818 (+ 2125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:37:08.933412 (+ 594us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:37:08.933418 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:37:08.933424 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:37:08.933596 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:37:08.934040 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:37:08.934044 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:37:08.934045 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:37:08.934684 (+ 639us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:37:08.935712 (+ 1028us) server_negotiation.cc:1035] Waiting for connection context
0809 09:37:08.935903 (+ 191us) server_negotiation.cc:299] Negotiation successful
0809 09:37:08.936139 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":356,"thread_start_us":158,"threads_started":1}
I20260809 09:37:08.938223 20705 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 09:37:08.940589 20982 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 09:37:08.947578 20705 catalog_manager.cc:1347] Generated new cluster ID: a69276db27674a758d85bfb391ebff62
I20260809 09:37:08.947657 20705 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 09:37:08.951956 20689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:37:08.942637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37401 (local address 127.20.125.190:43101)
0809 09:37:08.942764 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0809 09:37:08.942769 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 09:37:08.943028 (+ 259us) server_negotiation.cc:407] Connection header received
0809 09:37:08.943149 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:37:08.943152 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:37:08.943199 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:37:08.943283 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:37:08.944291 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:37:08.945127 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:37:08.946319 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:37:08.946772 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:37:08.946887 (+ 115us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:37:08.948479 (+ 1592us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:37:08.948483 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:37:08.948486 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:37:08.948519 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:37:08.950059 (+ 1540us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:37:08.950611 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:37:08.950614 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:37:08.950616 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:37:08.950701 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:37:08.951041 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:37:08.951044 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:37:08.951046 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:37:08.951227 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:37:08.951390 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0809 09:37:08.951682 (+ 292us) server_negotiation.cc:299] Negotiation successful
0809 09:37:08.951775 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260809 09:37:08.975303 20705 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 09:37:08.976686 20705 catalog_manager.cc:1504] Loading token signing keys...
I20260809 09:37:08.980859 20705 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b84d03dbf5564fe3bdef8d6d8311ab0a: Generated new TSK 0
I20260809 09:37:08.981714 20705 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 09:37:08.988512 20982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu with pid 20573
W20260809 09:37:09.002044 20555 connection.cc:535] client connection to 127.20.125.190:43101 recv error: Network error: failed to read from TLS socket (remote: 127.20.125.190:43101): Connection reset by peer (error 104)
2026-08-09T09:37:08Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1710 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786267941944493-20982-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 09:37:09 dist-test-slave-7kzw krb5kdc[20735](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 09:37:09 dist-test-slave-7kzw krb5kdc[20735](info): set up 2 sockets
Aug 09 09:37:09 dist-test-slave-7kzw krb5kdc[20735](info): commencing operation
krb5kdc: starting...
W20260809 09:37:10.224649 20982 mini_kdc.cc:121] Time spent starting KDC: real 1.208s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786267941944493-20982-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786267941944493-20982-0/krb5kdc/test-user.keytab.
Aug 09 09:37:10 dist-test-slave-7kzw krb5kdc[20735](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268230, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T09:37:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T09:37:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.125.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.125.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.125.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.190.keytab.
Entry for principal kudu/127.20.125.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.190.keytab.
I20260809 09:37:10.361640 20982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
/tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786267941944493-20982-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786267941944493-20982-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786267941944493-20982-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786267941944493-20982-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.125.190:41623
--webserver_interface=127.20.125.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.190.keytab
--principal=kudu/127.20.125.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.125.148:34439
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.125.190:41623
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786267941944493-20982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786267941944493-20982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786267941944493-20982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 09:37:10.475997 20750 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 09:37:10.476276 20750 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 09:37:10.476347 20750 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 09:37:10.480374 20750 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 09:37:10.480433 20750 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 09:37:10.480454 20750 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 09:37:10.480470 20750 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 09:37:10.480485 20750 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 09:37:10.485827 20750 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.125.148:34439
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786267941944493-20982-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786267941944493-20982-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.125.190:41623
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.190.keytab
--principal=kudu/127.20.125.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.125.190:41623
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786267941944493-20982-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.125.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20750
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786267941944493-20982-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 08:38:55 UTC on ccycloud.quasar-njjleu.root.comops.site
build id 4575935
I20260809 09:37:10.486958 20750 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 09:37:10.488049 20750 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 09:37:10.497087 20764 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:37:10.497222 20758 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:37:10.497236 20759 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:37:10.497407 20750 server_base.cc:1034] running on GCE node
I20260809 09:37:10.498178 20750 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 09:37:10.499408 20750 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 09:37:10.500590 20750 hybrid_clock.cc:648] HybridClock initialized: now 1786268230500572 us; error 38 us; skew 500 ppm
Aug 09 09:37:10 dist-test-slave-7kzw krb5kdc[20735](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268230, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 09:37:10.503815 20750 init.cc:377] Logged in from keytab as kudu/127.20.125.190@KRBTEST.COM (short username kudu)
I20260809 09:37:10.504804 20750 webserver.cc:460] Webserver started at http://127.20.125.190:41625/ using document root <none> and password file <none>
I20260809 09:37:10.505399 20750 fs_manager.cc:359] Metadata directory not provided
I20260809 09:37:10.505468 20750 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 09:37:10.505704 20750 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 09:37:10.507915 20750 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786267941944493-20982-0/minicluster-data/master-0/data/instance:
uuid: "62e802e4253443498b722d2a6f857250"
format_stamp: "Formatted at 2026-08-09 09:37:10 on dist-test-slave-7kzw"
server_key: "412ec747e75fb4cae9a3b5692b5a09e9"
server_key_iv: "a3c6235fa70a3f905166cc1f9ec671bb"
server_key_version: "encryptionkey@0"
I20260809 09:37:10.508522 20750 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786267941944493-20982-0/minicluster-data/master-0/wal/instance:
uuid: "62e802e4253443498b722d2a6f857250"
format_stamp: "Formatted at 2026-08-09 09:37:10 on dist-test-slave-7kzw"
server_key: "412ec747e75fb4cae9a3b5692b5a09e9"
server_key_iv: "a3c6235fa70a3f905166cc1f9ec671bb"
server_key_version: "encryptionkey@0"
I20260809 09:37:10.512804 20750 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 09:37:10.515393 20779 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 09:37:10.516709 20750 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 09:37:10.516883 20750 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786267941944493-20982-0/minicluster-data/master-0/data,/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786267941944493-20982-0/minicluster-data/master-0/wal
uuid: "62e802e4253443498b722d2a6f857250"
format_stamp: "Formatted at 2026-08-09 09:37:10 on dist-test-slave-7kzw"
server_key: "412ec747e75fb4cae9a3b5692b5a09e9"
server_key_iv: "a3c6235fa70a3f905166cc1f9ec671bb"
server_key_version: "encryptionkey@0"
I20260809 09:37:10.517020 20750 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786267941944493-20982-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786267941944493-20982-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786267941944493-20982-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 09:37:10.552229 20750 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 09:37:10.555820 20750 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 09:37:10.556039 20750 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 09:37:10.563640 20750 rpc_server.cc:307] RPC server started. Bound to: 127.20.125.190:41623
I20260809 09:37:10.563665 20858 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.125.190:41623 every 8 connection(s)
I20260809 09:37:10.564625 20750 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786267941944493-20982-0/minicluster-data/master-0/data/info.pb
I20260809 09:37:10.567534 20859 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 09:37:10.569121 20982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu as pid 20750
I20260809 09:37:10.569270 20982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786267941944493-20982-0/minicluster-data/master-0/wal/instance
I20260809 09:37:10.569597 20982 external_mini_cluster.cc:1428] Setting key 6b04ed6dcd759ee0c3899f43017023c3
I20260809 09:37:10.573896 20859 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 62e802e4253443498b722d2a6f857250: Bootstrap starting.
I20260809 09:37:10.576244 20859 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 62e802e4253443498b722d2a6f857250: Neither blocks nor log segments found. Creating new log.
I20260809 09:37:10.577013 20859 log.cc:824] T 00000000000000000000000000000000 P 62e802e4253443498b722d2a6f857250: Log is configured to *not* fsync() on all Append() calls
I20260809 09:37:10.579078 20864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:37:10.571148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60125 (local address 127.20.125.190:41623)
0809 09:37:10.571626 (+ 478us) server_negotiation.cc:206] Beginning negotiation
0809 09:37:10.571646 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0809 09:37:10.571686 (+ 40us) server_negotiation.cc:407] Connection header received
0809 09:37:10.572335 (+ 649us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:37:10.572352 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:37:10.572684 (+ 332us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:37:10.573018 (+ 334us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:37:10.573439 (+ 421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:37:10.574549 (+ 1110us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:37:10.575491 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:37:10.575983 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:37:10.576175 (+ 192us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:37:10.577167 (+ 992us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:37:10.577175 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:37:10.577193 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 09:37:10.577697 (+ 504us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:37:10.577771 (+ 74us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 09:37:10.577806 (+ 35us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:37:10.577946 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0809 09:37:10.578147 (+ 201us) server_negotiation.cc:299] Negotiation successful
0809 09:37:10.578295 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":323,"thread_start_us":143,"threads_started":1}
I20260809 09:37:10.579324 20859 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 62e802e4253443498b722d2a6f857250: No bootstrap required, opened a new log
I20260809 09:37:10.582141 20982 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 09:37:10.583396 20859 raft_consensus.cc:348] T 00000000000000000000000000000000 P 62e802e4253443498b722d2a6f857250 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62e802e4253443498b722d2a6f857250" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 41623 } }
I20260809 09:37:10.583689 20859 raft_consensus.cc:374] T 00000000000000000000000000000000 P 62e802e4253443498b722d2a6f857250 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 09:37:10.583751 20859 raft_consensus.cc:729] T 00000000000000000000000000000000 P 62e802e4253443498b722d2a6f857250 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 62e802e4253443498b722d2a6f857250, State: Initialized, Role: FOLLOWER
I20260809 09:37:10.584203 20859 consensus_queue.cc:260] T 00000000000000000000000000000000 P 62e802e4253443498b722d2a6f857250 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62e802e4253443498b722d2a6f857250" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 41623 } }
I20260809 09:37:10.584367 20859 raft_consensus.cc:388] T 00000000000000000000000000000000 P 62e802e4253443498b722d2a6f857250 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 09:37:10.584462 20859 raft_consensus.cc:482] T 00000000000000000000000000000000 P 62e802e4253443498b722d2a6f857250 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 09:37:10.584596 20859 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 62e802e4253443498b722d2a6f857250 [term 0 FOLLOWER]: Advancing to term 1
I20260809 09:37:10.585520 20859 raft_consensus.cc:504] T 00000000000000000000000000000000 P 62e802e4253443498b722d2a6f857250 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62e802e4253443498b722d2a6f857250" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 41623 } }
I20260809 09:37:10.585860 20859 leader_election.cc:302] T 00000000000000000000000000000000 P 62e802e4253443498b722d2a6f857250 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 62e802e4253443498b722d2a6f857250; no voters:
I20260809 09:37:10.586212 20859 leader_election.cc:288] T 00000000000000000000000000000000 P 62e802e4253443498b722d2a6f857250 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 09:37:10.586304 20874 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 62e802e4253443498b722d2a6f857250 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 09:37:10.586580 20874 raft_consensus.cc:686] T 00000000000000000000000000000000 P 62e802e4253443498b722d2a6f857250 [term 1 LEADER]: Becoming Leader. State: Replica: 62e802e4253443498b722d2a6f857250, State: Running, Role: LEADER
I20260809 09:37:10.586880 20874 consensus_queue.cc:237] T 00000000000000000000000000000000 P 62e802e4253443498b722d2a6f857250 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62e802e4253443498b722d2a6f857250" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 41623 } }
I20260809 09:37:10.587561 20859 sys_catalog.cc:564] T 00000000000000000000000000000000 P 62e802e4253443498b722d2a6f857250 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 09:37:10.588137 20874 sys_catalog.cc:455] T 00000000000000000000000000000000 P 62e802e4253443498b722d2a6f857250 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 62e802e4253443498b722d2a6f857250. Latest consensus state: current_term: 1 leader_uuid: "62e802e4253443498b722d2a6f857250" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62e802e4253443498b722d2a6f857250" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 41623 } } }
I20260809 09:37:10.588270 20874 sys_catalog.cc:458] T 00000000000000000000000000000000 P 62e802e4253443498b722d2a6f857250 [sys.catalog]: This master's current role is: LEADER
I20260809 09:37:10.588690 20880 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 09:37:10.588918 20875 sys_catalog.cc:455] T 00000000000000000000000000000000 P 62e802e4253443498b722d2a6f857250 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "62e802e4253443498b722d2a6f857250" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62e802e4253443498b722d2a6f857250" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 41623 } } }
I20260809 09:37:10.589072 20875 sys_catalog.cc:458] T 00000000000000000000000000000000 P 62e802e4253443498b722d2a6f857250 [sys.catalog]: This master's current role is: LEADER
I20260809 09:37:10.589280 20864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:37:10.584422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54067 (local address 127.20.125.190:41623)
0809 09:37:10.584526 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0809 09:37:10.584530 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 09:37:10.584844 (+ 314us) server_negotiation.cc:407] Connection header received
0809 09:37:10.584979 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:37:10.584982 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:37:10.585031 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:37:10.585121 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:37:10.585390 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:37:10.585946 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:37:10.587113 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:37:10.587456 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:37:10.587579 (+ 123us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:37:10.588561 (+ 982us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:37:10.588580 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:37:10.588582 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 09:37:10.588595 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:37:10.588628 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 09:37:10.588646 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:37:10.588733 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0809 09:37:10.589068 (+ 335us) server_negotiation.cc:299] Negotiation successful
0809 09:37:10.589103 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260809 09:37:10.592139 20880 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 09:37:10.597513 20880 catalog_manager.cc:1347] Generated new cluster ID: 3024a50ddee74c9ba895dce01c69a180
I20260809 09:37:10.597581 20880 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 09:37:10.612102 20880 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 09:37:10.613029 20880 catalog_manager.cc:1504] Loading token signing keys...
I20260809 09:37:10.619143 20880 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 62e802e4253443498b722d2a6f857250: Generated new TSK 0
I20260809 09:37:10.619767 20880 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 09:37:10.624405 20982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu with pid 20750
W20260809 09:37:10.634918 20733 connection.cc:535] client connection to 127.20.125.190:41623 recv error: Network error: failed to read from TLS socket (remote: 127.20.125.190:41623): Connection reset by peer (error 104)
2026-08-09T09:37:10Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1632 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786267941944493-20982-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 09:37:10 dist-test-slave-7kzw krb5kdc[20904](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 09:37:10 dist-test-slave-7kzw krb5kdc[20904](info): set up 2 sockets
Aug 09 09:37:10 dist-test-slave-7kzw krb5kdc[20904](info): commencing operation
krb5kdc: starting...
W20260809 09:37:11.929045 20982 mini_kdc.cc:121] Time spent starting KDC: real 1.280s user 0.001s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786267941944493-20982-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786267941944493-20982-0/krb5kdc/test-user.keytab.
Aug 09 09:37:12 dist-test-slave-7kzw krb5kdc[20904](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268232, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T09:37:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T09:37:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.125.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.125.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.125.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.190.keytab.
Entry for principal kudu/127.20.125.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.190.keytab.
I20260809 09:37:12.066872 20982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
/tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786267941944493-20982-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786267941944493-20982-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786267941944493-20982-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786267941944493-20982-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.125.190:35917
--webserver_interface=127.20.125.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.190.keytab
--principal=kudu/127.20.125.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.125.148:39821
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.125.190:35917
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786267941944493-20982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786267941944493-20982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786267941944493-20982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 09:37:12.178609 20919 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 09:37:12.178948 20919 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 09:37:12.179028 20919 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 09:37:12.183041 20919 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 09:37:12.183147 20919 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 09:37:12.183193 20919 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 09:37:12.183254 20919 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 09:37:12.183295 20919 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 09:37:12.188539 20919 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.125.148:39821
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786267941944493-20982-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786267941944493-20982-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.125.190:35917
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.190.keytab
--principal=kudu/127.20.125.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.125.190:35917
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786267941944493-20982-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.125.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20919
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786267941944493-20982-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 08:38:55 UTC on ccycloud.quasar-njjleu.root.comops.site
build id 4575935
I20260809 09:37:12.189590 20919 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 09:37:12.190676 20919 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 09:37:12.198894 20929 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:37:12.198904 20932 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:37:12.198908 20928 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:37:12.199376 20919 server_base.cc:1034] running on GCE node
I20260809 09:37:12.199880 20919 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 09:37:12.200924 20919 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 09:37:12.202092 20919 hybrid_clock.cc:648] HybridClock initialized: now 1786268232202073 us; error 30 us; skew 500 ppm
Aug 09 09:37:12 dist-test-slave-7kzw krb5kdc[20904](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268232, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 09:37:12.205013 20919 init.cc:377] Logged in from keytab as kudu/127.20.125.190@KRBTEST.COM (short username kudu)
I20260809 09:37:12.205919 20919 webserver.cc:460] Webserver started at http://127.20.125.190:35633/ using document root <none> and password file <none>
I20260809 09:37:12.206431 20919 fs_manager.cc:359] Metadata directory not provided
I20260809 09:37:12.206548 20919 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 09:37:12.206727 20919 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 09:37:12.208709 20919 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786267941944493-20982-0/minicluster-data/master-0/data/instance:
uuid: "ce16c9dfc08e476d998c3d16241ea7c6"
format_stamp: "Formatted at 2026-08-09 09:37:12 on dist-test-slave-7kzw"
server_key: "54d0453db1c2332b81e639a136f58219"
server_key_iv: "407c75ce8604c2b998ba0a55de5d650a"
server_key_version: "encryptionkey@0"
I20260809 09:37:12.209233 20919 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786267941944493-20982-0/minicluster-data/master-0/wal/instance:
uuid: "ce16c9dfc08e476d998c3d16241ea7c6"
format_stamp: "Formatted at 2026-08-09 09:37:12 on dist-test-slave-7kzw"
server_key: "54d0453db1c2332b81e639a136f58219"
server_key_iv: "407c75ce8604c2b998ba0a55de5d650a"
server_key_version: "encryptionkey@0"
I20260809 09:37:12.212841 20919 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.003s
I20260809 09:37:12.215480 20942 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 09:37:12.216856 20919 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 09:37:12.217044 20919 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786267941944493-20982-0/minicluster-data/master-0/data,/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786267941944493-20982-0/minicluster-data/master-0/wal
uuid: "ce16c9dfc08e476d998c3d16241ea7c6"
format_stamp: "Formatted at 2026-08-09 09:37:12 on dist-test-slave-7kzw"
server_key: "54d0453db1c2332b81e639a136f58219"
server_key_iv: "407c75ce8604c2b998ba0a55de5d650a"
server_key_version: "encryptionkey@0"
I20260809 09:37:12.217180 20919 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786267941944493-20982-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786267941944493-20982-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786267941944493-20982-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 09:37:12.222340 20919 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 09:37:12.225543 20919 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 09:37:12.225776 20919 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 09:37:12.234438 20919 rpc_server.cc:307] RPC server started. Bound to: 127.20.125.190:35917
I20260809 09:37:12.234454 21021 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.125.190:35917 every 8 connection(s)
I20260809 09:37:12.235607 20919 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786267941944493-20982-0/minicluster-data/master-0/data/info.pb
I20260809 09:37:12.238549 21025 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 09:37:12.243481 20982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu as pid 20919
I20260809 09:37:12.243706 20982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786267941944493-20982-0/minicluster-data/master-0/wal/instance
I20260809 09:37:12.244005 20982 external_mini_cluster.cc:1428] Setting key 7efa6f179be81901abcc138b1cdfa833
I20260809 09:37:12.244391 21025 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ce16c9dfc08e476d998c3d16241ea7c6: Bootstrap starting.
I20260809 09:37:12.246954 21025 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ce16c9dfc08e476d998c3d16241ea7c6: Neither blocks nor log segments found. Creating new log.
I20260809 09:37:12.247805 21025 log.cc:824] T 00000000000000000000000000000000 P ce16c9dfc08e476d998c3d16241ea7c6: Log is configured to *not* fsync() on all Append() calls
I20260809 09:37:12.250108 21025 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ce16c9dfc08e476d998c3d16241ea7c6: No bootstrap required, opened a new log
I20260809 09:37:12.250087 21030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:37:12.245659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58455 (local address 127.20.125.190:35917)
0809 09:37:12.246180 (+ 521us) server_negotiation.cc:206] Beginning negotiation
0809 09:37:12.246197 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0809 09:37:12.246239 (+ 42us) server_negotiation.cc:407] Connection header received
0809 09:37:12.247051 (+ 812us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:37:12.247081 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:37:12.247422 (+ 341us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:37:12.247742 (+ 320us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:37:12.248743 (+ 1001us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:37:12.248750 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:37:12.248772 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 09:37:12.248787 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:37:12.248852 (+ 65us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 09:37:12.248877 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:37:12.248982 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0809 09:37:12.249118 (+ 136us) server_negotiation.cc:299] Negotiation successful
0809 09:37:12.249300 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":320,"thread_start_us":178,"threads_started":1}
I20260809 09:37:12.253504 20982 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 09:37:12.254333 21025 raft_consensus.cc:348] T 00000000000000000000000000000000 P ce16c9dfc08e476d998c3d16241ea7c6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce16c9dfc08e476d998c3d16241ea7c6" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 35917 } }
I20260809 09:37:12.254659 21025 raft_consensus.cc:374] T 00000000000000000000000000000000 P ce16c9dfc08e476d998c3d16241ea7c6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 09:37:12.254724 21025 raft_consensus.cc:729] T 00000000000000000000000000000000 P ce16c9dfc08e476d998c3d16241ea7c6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce16c9dfc08e476d998c3d16241ea7c6, State: Initialized, Role: FOLLOWER
I20260809 09:37:12.255259 21025 consensus_queue.cc:260] T 00000000000000000000000000000000 P ce16c9dfc08e476d998c3d16241ea7c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce16c9dfc08e476d998c3d16241ea7c6" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 35917 } }
I20260809 09:37:12.255435 21025 raft_consensus.cc:388] T 00000000000000000000000000000000 P ce16c9dfc08e476d998c3d16241ea7c6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 09:37:12.255505 21025 raft_consensus.cc:482] T 00000000000000000000000000000000 P ce16c9dfc08e476d998c3d16241ea7c6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 09:37:12.255597 21025 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ce16c9dfc08e476d998c3d16241ea7c6 [term 0 FOLLOWER]: Advancing to term 1
I20260809 09:37:12.256918 21025 raft_consensus.cc:504] T 00000000000000000000000000000000 P ce16c9dfc08e476d998c3d16241ea7c6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce16c9dfc08e476d998c3d16241ea7c6" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 35917 } }
I20260809 09:37:12.257308 21025 leader_election.cc:302] T 00000000000000000000000000000000 P ce16c9dfc08e476d998c3d16241ea7c6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ce16c9dfc08e476d998c3d16241ea7c6; no voters:
I20260809 09:37:12.257717 21025 leader_election.cc:288] T 00000000000000000000000000000000 P ce16c9dfc08e476d998c3d16241ea7c6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 09:37:12.257827 21042 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ce16c9dfc08e476d998c3d16241ea7c6 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 09:37:12.258069 21042 raft_consensus.cc:686] T 00000000000000000000000000000000 P ce16c9dfc08e476d998c3d16241ea7c6 [term 1 LEADER]: Becoming Leader. State: Replica: ce16c9dfc08e476d998c3d16241ea7c6, State: Running, Role: LEADER
I20260809 09:37:12.258113 21030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:37:12.256283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44487 (local address 127.20.125.190:35917)
0809 09:37:12.256660 (+ 377us) server_negotiation.cc:206] Beginning negotiation
0809 09:37:12.256697 (+ 37us) server_negotiation.cc:399] Waiting for connection header
0809 09:37:12.256718 (+ 21us) server_negotiation.cc:407] Connection header received
0809 09:37:12.256777 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:37:12.256780 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:37:12.256812 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:37:12.256883 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:37:12.257619 (+ 736us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:37:12.257622 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:37:12.257624 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 09:37:12.257635 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:37:12.257662 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 09:37:12.257696 (+ 34us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:37:12.257798 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0809 09:37:12.257945 (+ 147us) server_negotiation.cc:299] Negotiation successful
0809 09:37:12.257995 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":257}
I20260809 09:37:12.258436 21042 consensus_queue.cc:237] T 00000000000000000000000000000000 P ce16c9dfc08e476d998c3d16241ea7c6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce16c9dfc08e476d998c3d16241ea7c6" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 35917 } }
I20260809 09:37:12.259148 21025 sys_catalog.cc:564] T 00000000000000000000000000000000 P ce16c9dfc08e476d998c3d16241ea7c6 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 09:37:12.259848 21045 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce16c9dfc08e476d998c3d16241ea7c6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ce16c9dfc08e476d998c3d16241ea7c6. Latest consensus state: current_term: 1 leader_uuid: "ce16c9dfc08e476d998c3d16241ea7c6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce16c9dfc08e476d998c3d16241ea7c6" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 35917 } } }
I20260809 09:37:12.260008 21045 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce16c9dfc08e476d998c3d16241ea7c6 [sys.catalog]: This master's current role is: LEADER
I20260809 09:37:12.260337 21044 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce16c9dfc08e476d998c3d16241ea7c6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ce16c9dfc08e476d998c3d16241ea7c6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce16c9dfc08e476d998c3d16241ea7c6" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 35917 } } }
I20260809 09:37:12.260445 21044 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce16c9dfc08e476d998c3d16241ea7c6 [sys.catalog]: This master's current role is: LEADER
I20260809 09:37:12.260493 21052 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 09:37:12.263597 21052 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 09:37:12.269829 21052 catalog_manager.cc:1347] Generated new cluster ID: 0a038e7962f342f7b0182fb098b38671
I20260809 09:37:12.269909 21052 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 09:37:12.279024 21052 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 09:37:12.279340 21052 catalog_manager.cc:1504] Loading token signing keys...
I20260809 09:37:12.286595 21052 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ce16c9dfc08e476d998c3d16241ea7c6: Generated new TSK 0
I20260809 09:37:12.287247 21052 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 09:37:12.301209 20982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu with pid 20919
2026-08-09T09:37:12Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1683 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786267941944493-20982-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 09:37:12 dist-test-slave-7kzw krb5kdc[21072](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 09:37:12 dist-test-slave-7kzw krb5kdc[21072](info): set up 2 sockets
Aug 09 09:37:12 dist-test-slave-7kzw krb5kdc[21072](info): commencing operation
krb5kdc: starting...
W20260809 09:37:13.545836 20982 mini_kdc.cc:121] Time spent starting KDC: real 1.213s user 0.003s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786267941944493-20982-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786267941944493-20982-0/krb5kdc/test-user.keytab.
Aug 09 09:37:13 dist-test-slave-7kzw krb5kdc[21072](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268233, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T09:37:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T09:37:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.125.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.125.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.125.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.190.keytab.
Entry for principal kudu/127.20.125.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.190.keytab.
I20260809 09:37:13.677623 20982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
/tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786267941944493-20982-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786267941944493-20982-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786267941944493-20982-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786267941944493-20982-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.125.190:40301
--webserver_interface=127.20.125.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.190.keytab
--principal=kudu/127.20.125.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.125.148:41119
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.125.190:40301
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786267941944493-20982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786267941944493-20982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786267941944493-20982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 09:37:13.792898 21089 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 09:37:13.793165 21089 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 09:37:13.793247 21089 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 09:37:13.797676 21089 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 09:37:13.797756 21089 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 09:37:13.797780 21089 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 09:37:13.797798 21089 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 09:37:13.797816 21089 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 09:37:13.803747 21089 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.125.148:41119
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786267941944493-20982-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786267941944493-20982-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.125.190:40301
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.190.keytab
--principal=kudu/127.20.125.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.125.190:40301
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786267941944493-20982-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.125.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21089
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786267941944493-20982-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 08:38:55 UTC on ccycloud.quasar-njjleu.root.comops.site
build id 4575935
I20260809 09:37:13.804832 21089 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 09:37:13.805891 21089 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 09:37:13.814807 21101 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:37:13.814944 21096 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:37:13.814937 21099 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:37:13.815771 21089 server_base.cc:1034] running on GCE node
I20260809 09:37:13.816386 21089 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 09:37:13.817505 21089 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 09:37:13.818673 21089 hybrid_clock.cc:648] HybridClock initialized: now 1786268233818661 us; error 27 us; skew 500 ppm
Aug 09 09:37:13 dist-test-slave-7kzw krb5kdc[21072](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268233, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 09:37:13.821434 21089 init.cc:377] Logged in from keytab as kudu/127.20.125.190@KRBTEST.COM (short username kudu)
I20260809 09:37:13.822335 21089 webserver.cc:460] Webserver started at http://127.20.125.190:37993/ using document root <none> and password file <none>
I20260809 09:37:13.822952 21089 fs_manager.cc:359] Metadata directory not provided
I20260809 09:37:13.823050 21089 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 09:37:13.823285 21089 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 09:37:13.825377 21089 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786267941944493-20982-0/minicluster-data/master-0/data/instance:
uuid: "4584412cf90f4e3ea7d78052b2ddb4b7"
format_stamp: "Formatted at 2026-08-09 09:37:13 on dist-test-slave-7kzw"
server_key: "e370f5b9d013b37a00cd78372a8fcf3b"
server_key_iv: "2f1708a267395bc86c7b646e3cf5ca27"
server_key_version: "encryptionkey@0"
I20260809 09:37:13.825990 21089 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786267941944493-20982-0/minicluster-data/master-0/wal/instance:
uuid: "4584412cf90f4e3ea7d78052b2ddb4b7"
format_stamp: "Formatted at 2026-08-09 09:37:13 on dist-test-slave-7kzw"
server_key: "e370f5b9d013b37a00cd78372a8fcf3b"
server_key_iv: "2f1708a267395bc86c7b646e3cf5ca27"
server_key_version: "encryptionkey@0"
I20260809 09:37:13.830035 21089 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260809 09:37:13.832739 21110 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 09:37:13.834194 21089 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 09:37:13.834381 21089 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786267941944493-20982-0/minicluster-data/master-0/data,/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786267941944493-20982-0/minicluster-data/master-0/wal
uuid: "4584412cf90f4e3ea7d78052b2ddb4b7"
format_stamp: "Formatted at 2026-08-09 09:37:13 on dist-test-slave-7kzw"
server_key: "e370f5b9d013b37a00cd78372a8fcf3b"
server_key_iv: "2f1708a267395bc86c7b646e3cf5ca27"
server_key_version: "encryptionkey@0"
I20260809 09:37:13.834583 21089 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786267941944493-20982-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786267941944493-20982-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786267941944493-20982-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 09:37:13.848502 21089 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 09:37:13.851778 21089 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 09:37:13.851991 21089 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 09:37:13.860226 21089 rpc_server.cc:307] RPC server started. Bound to: 127.20.125.190:40301
I20260809 09:37:13.860283 21187 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.125.190:40301 every 8 connection(s)
I20260809 09:37:13.861606 21089 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786267941944493-20982-0/minicluster-data/master-0/data/info.pb
I20260809 09:37:13.864687 20982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu as pid 21089
I20260809 09:37:13.864854 20982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786267941944493-20982-0/minicluster-data/master-0/wal/instance
I20260809 09:37:13.865211 20982 external_mini_cluster.cc:1428] Setting key c95adf93fa3999502ae7521d00a5e511
I20260809 09:37:13.865598 21188 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 09:37:13.872244 21188 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4584412cf90f4e3ea7d78052b2ddb4b7: Bootstrap starting.
I20260809 09:37:13.874614 21188 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4584412cf90f4e3ea7d78052b2ddb4b7: Neither blocks nor log segments found. Creating new log.
Aug 09 09:37:13 dist-test-slave-7kzw krb5kdc[21072](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268233, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.125.190@KRBTEST.COM
I20260809 09:37:13.875590 21188 log.cc:824] T 00000000000000000000000000000000 P 4584412cf90f4e3ea7d78052b2ddb4b7: Log is configured to *not* fsync() on all Append() calls
I20260809 09:37:13.877836 21188 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4584412cf90f4e3ea7d78052b2ddb4b7: No bootstrap required, opened a new log
I20260809 09:37:13.880330 21191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:37:13.866802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39829 (local address 127.20.125.190:40301)
0809 09:37:13.867403 (+ 601us) server_negotiation.cc:206] Beginning negotiation
0809 09:37:13.867428 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0809 09:37:13.867486 (+ 58us) server_negotiation.cc:407] Connection header received
0809 09:37:13.868223 (+ 737us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:37:13.868240 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:37:13.868590 (+ 350us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:37:13.868930 (+ 340us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:37:13.869804 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:37:13.870887 (+ 1083us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:37:13.871752 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:37:13.872235 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:37:13.872473 (+ 238us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:37:13.875505 (+ 3032us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:37:13.875510 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:37:13.875523 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:37:13.875552 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:37:13.877338 (+ 1786us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:37:13.877834 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:37:13.877840 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:37:13.877846 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:37:13.878022 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:37:13.878333 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:37:13.878337 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:37:13.878338 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:37:13.878821 (+ 483us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:37:13.879044 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0809 09:37:13.879289 (+ 245us) server_negotiation.cc:299] Negotiation successful
0809 09:37:13.879608 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":378,"thread_start_us":160,"threads_started":1}
I20260809 09:37:13.881582 21188 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4584412cf90f4e3ea7d78052b2ddb4b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4584412cf90f4e3ea7d78052b2ddb4b7" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 40301 } }
I20260809 09:37:13.881809 21188 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4584412cf90f4e3ea7d78052b2ddb4b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 09:37:13.881875 21188 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4584412cf90f4e3ea7d78052b2ddb4b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4584412cf90f4e3ea7d78052b2ddb4b7, State: Initialized, Role: FOLLOWER
I20260809 09:37:13.882346 21188 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4584412cf90f4e3ea7d78052b2ddb4b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4584412cf90f4e3ea7d78052b2ddb4b7" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 40301 } }
I20260809 09:37:13.882570 21188 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4584412cf90f4e3ea7d78052b2ddb4b7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 09:37:13.882661 21188 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4584412cf90f4e3ea7d78052b2ddb4b7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 09:37:13.882814 21188 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4584412cf90f4e3ea7d78052b2ddb4b7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 09:37:13.883354 20982 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 09:37:13.883903 21188 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4584412cf90f4e3ea7d78052b2ddb4b7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4584412cf90f4e3ea7d78052b2ddb4b7" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 40301 } }
I20260809 09:37:13.884248 21188 leader_election.cc:302] T 00000000000000000000000000000000 P 4584412cf90f4e3ea7d78052b2ddb4b7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4584412cf90f4e3ea7d78052b2ddb4b7; no voters:
I20260809 09:37:13.884610 21188 leader_election.cc:288] T 00000000000000000000000000000000 P 4584412cf90f4e3ea7d78052b2ddb4b7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 09:37:13.884696 21195 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4584412cf90f4e3ea7d78052b2ddb4b7 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 09:37:13.884909 21195 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4584412cf90f4e3ea7d78052b2ddb4b7 [term 1 LEADER]: Becoming Leader. State: Replica: 4584412cf90f4e3ea7d78052b2ddb4b7, State: Running, Role: LEADER
I20260809 09:37:13.885210 21195 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4584412cf90f4e3ea7d78052b2ddb4b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4584412cf90f4e3ea7d78052b2ddb4b7" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 40301 } }
I20260809 09:37:13.885926 21188 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4584412cf90f4e3ea7d78052b2ddb4b7 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 09:37:13.886970 21201 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4584412cf90f4e3ea7d78052b2ddb4b7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4584412cf90f4e3ea7d78052b2ddb4b7. Latest consensus state: current_term: 1 leader_uuid: "4584412cf90f4e3ea7d78052b2ddb4b7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4584412cf90f4e3ea7d78052b2ddb4b7" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 40301 } } }
I20260809 09:37:13.887008 21200 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4584412cf90f4e3ea7d78052b2ddb4b7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4584412cf90f4e3ea7d78052b2ddb4b7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4584412cf90f4e3ea7d78052b2ddb4b7" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 40301 } } }
I20260809 09:37:13.887107 21201 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4584412cf90f4e3ea7d78052b2ddb4b7 [sys.catalog]: This master's current role is: LEADER
I20260809 09:37:13.887167 21200 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4584412cf90f4e3ea7d78052b2ddb4b7 [sys.catalog]: This master's current role is: LEADER
I20260809 09:37:13.887513 21220 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 09:37:13.890818 21220 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 09:37:13.894869 21191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:37:13.885904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40339 (local address 127.20.125.190:40301)
0809 09:37:13.886029 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0809 09:37:13.886034 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 09:37:13.886046 (+ 12us) server_negotiation.cc:407] Connection header received
0809 09:37:13.886076 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:37:13.886078 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:37:13.886116 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:37:13.886187 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:37:13.887070 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:37:13.887869 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:37:13.889094 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:37:13.889422 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:37:13.889497 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:37:13.891449 (+ 1952us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:37:13.891453 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:37:13.891455 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:37:13.891480 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:37:13.893155 (+ 1675us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:37:13.893574 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:37:13.893577 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:37:13.893579 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:37:13.893667 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:37:13.893968 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:37:13.893972 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:37:13.893973 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:37:13.894156 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:37:13.894299 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0809 09:37:13.894574 (+ 275us) server_negotiation.cc:299] Negotiation successful
0809 09:37:13.894704 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260809 09:37:13.897238 21220 catalog_manager.cc:1347] Generated new cluster ID: f6e8e2a1b53f4490a0f56f846136f61c
I20260809 09:37:13.897305 21220 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 09:37:13.909902 21220 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 09:37:13.910997 21220 catalog_manager.cc:1504] Loading token signing keys...
I20260809 09:37:13.920353 21220 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4584412cf90f4e3ea7d78052b2ddb4b7: Generated new TSK 0
I20260809 09:37:13.921144 21220 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 09:37:13.929402 20982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu with pid 21089
2026-08-09T09:37:13Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1622 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786267941944493-20982-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 09:37:13 dist-test-slave-7kzw krb5kdc[21239](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 09:37:13 dist-test-slave-7kzw krb5kdc[21239](info): set up 2 sockets
Aug 09 09:37:13 dist-test-slave-7kzw krb5kdc[21239](info): commencing operation
krb5kdc: starting...
W20260809 09:37:15.193193 20982 mini_kdc.cc:121] Time spent starting KDC: real 1.238s user 0.003s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786267941944493-20982-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786267941944493-20982-0/krb5kdc/test-user.keytab.
Aug 09 09:37:15 dist-test-slave-7kzw krb5kdc[21239](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268235, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T09:37:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T09:37:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.125.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.125.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.125.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.190.keytab.
Entry for principal kudu/127.20.125.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.190.keytab.
I20260809 09:37:15.327432 20982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
/tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786267941944493-20982-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786267941944493-20982-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786267941944493-20982-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786267941944493-20982-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.125.190:35605
--webserver_interface=127.20.125.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.190.keytab
--principal=kudu/127.20.125.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.125.148:45777
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.125.190:35605
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786267941944493-20982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786267941944493-20982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786267941944493-20982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 09:37:15.444216 21256 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 09:37:15.444486 21256 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 09:37:15.444537 21256 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 09:37:15.448603 21256 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 09:37:15.448673 21256 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 09:37:15.448694 21256 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 09:37:15.448714 21256 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 09:37:15.448774 21256 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 09:37:15.454030 21256 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.125.148:45777
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786267941944493-20982-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786267941944493-20982-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.125.190:35605
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.190.keytab
--principal=kudu/127.20.125.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.125.190:35605
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786267941944493-20982-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.125.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21256
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786267941944493-20982-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 08:38:55 UTC on ccycloud.quasar-njjleu.root.comops.site
build id 4575935
I20260809 09:37:15.455199 21256 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 09:37:15.456310 21256 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 09:37:15.465822 21263 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:37:15.465852 21264 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:37:15.465914 21256 server_base.cc:1034] running on GCE node
W20260809 09:37:15.465822 21266 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:37:15.466722 21256 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 09:37:15.467849 21256 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 09:37:15.469070 21256 hybrid_clock.cc:648] HybridClock initialized: now 1786268235469055 us; error 26 us; skew 500 ppm
Aug 09 09:37:15 dist-test-slave-7kzw krb5kdc[21239](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268235, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 09:37:15.472160 21256 init.cc:377] Logged in from keytab as kudu/127.20.125.190@KRBTEST.COM (short username kudu)
I20260809 09:37:15.473166 21256 webserver.cc:460] Webserver started at http://127.20.125.190:44275/ using document root <none> and password file <none>
I20260809 09:37:15.473804 21256 fs_manager.cc:359] Metadata directory not provided
I20260809 09:37:15.473905 21256 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 09:37:15.474161 21256 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 09:37:15.476341 21256 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786267941944493-20982-0/minicluster-data/master-0/data/instance:
uuid: "a925915099d44bbebfab53a6cd9ede0e"
format_stamp: "Formatted at 2026-08-09 09:37:15 on dist-test-slave-7kzw"
server_key: "7bebeb6bf6b8857471269d36c46830f6"
server_key_iv: "7352622fff1bee4aae44fe0fa289a037"
server_key_version: "encryptionkey@0"
I20260809 09:37:15.476928 21256 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786267941944493-20982-0/minicluster-data/master-0/wal/instance:
uuid: "a925915099d44bbebfab53a6cd9ede0e"
format_stamp: "Formatted at 2026-08-09 09:37:15 on dist-test-slave-7kzw"
server_key: "7bebeb6bf6b8857471269d36c46830f6"
server_key_iv: "7352622fff1bee4aae44fe0fa289a037"
server_key_version: "encryptionkey@0"
I20260809 09:37:15.480893 21256 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 09:37:15.483640 21273 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 09:37:15.485054 21256 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 09:37:15.485249 21256 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786267941944493-20982-0/minicluster-data/master-0/data,/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786267941944493-20982-0/minicluster-data/master-0/wal
uuid: "a925915099d44bbebfab53a6cd9ede0e"
format_stamp: "Formatted at 2026-08-09 09:37:15 on dist-test-slave-7kzw"
server_key: "7bebeb6bf6b8857471269d36c46830f6"
server_key_iv: "7352622fff1bee4aae44fe0fa289a037"
server_key_version: "encryptionkey@0"
I20260809 09:37:15.485380 21256 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786267941944493-20982-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786267941944493-20982-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786267941944493-20982-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 09:37:15.504956 21256 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 09:37:15.508502 21256 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 09:37:15.508729 21256 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 09:37:15.516140 21256 rpc_server.cc:307] RPC server started. Bound to: 127.20.125.190:35605
I20260809 09:37:15.516170 21345 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.125.190:35605 every 8 connection(s)
I20260809 09:37:15.517093 21256 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786267941944493-20982-0/minicluster-data/master-0/data/info.pb
I20260809 09:37:15.519817 21346 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 09:37:15.524708 20982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu as pid 21256
I20260809 09:37:15.524873 20982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786267941944493-20982-0/minicluster-data/master-0/wal/instance
I20260809 09:37:15.525224 20982 external_mini_cluster.cc:1428] Setting key 51c1c141dc92af5e5b0cb71cee421adc
I20260809 09:37:15.525729 21346 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a925915099d44bbebfab53a6cd9ede0e: Bootstrap starting.
I20260809 09:37:15.528267 21346 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a925915099d44bbebfab53a6cd9ede0e: Neither blocks nor log segments found. Creating new log.
I20260809 09:37:15.529004 21346 log.cc:824] T 00000000000000000000000000000000 P a925915099d44bbebfab53a6cd9ede0e: Log is configured to *not* fsync() on all Append() calls
I20260809 09:37:15.530967 21346 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a925915099d44bbebfab53a6cd9ede0e: No bootstrap required, opened a new log
I20260809 09:37:15.534601 21346 raft_consensus.cc:348] T 00000000000000000000000000000000 P a925915099d44bbebfab53a6cd9ede0e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a925915099d44bbebfab53a6cd9ede0e" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 35605 } }
I20260809 09:37:15.534884 21346 raft_consensus.cc:374] T 00000000000000000000000000000000 P a925915099d44bbebfab53a6cd9ede0e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 09:37:15.534983 21346 raft_consensus.cc:729] T 00000000000000000000000000000000 P a925915099d44bbebfab53a6cd9ede0e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a925915099d44bbebfab53a6cd9ede0e, State: Initialized, Role: FOLLOWER
I20260809 09:37:15.535346 21349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:37:15.526752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49427 (local address 127.20.125.190:35605)
0809 09:37:15.527244 (+ 492us) server_negotiation.cc:206] Beginning negotiation
0809 09:37:15.527261 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0809 09:37:15.527314 (+ 53us) server_negotiation.cc:407] Connection header received
0809 09:37:15.528015 (+ 701us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:37:15.528034 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:37:15.528432 (+ 398us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:37:15.528823 (+ 391us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:37:15.529278 (+ 455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:37:15.530309 (+ 1031us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:37:15.531420 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:37:15.531941 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:37:15.532134 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:37:15.533090 (+ 956us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:37:15.533096 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:37:15.533109 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 09:37:15.533516 (+ 407us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:37:15.533592 (+ 76us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 09:37:15.533618 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:37:15.533747 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0809 09:37:15.533976 (+ 229us) server_negotiation.cc:299] Negotiation successful
0809 09:37:15.534298 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":339,"thread_start_us":144,"threads_started":1}
I20260809 09:37:15.535460 21346 consensus_queue.cc:260] T 00000000000000000000000000000000 P a925915099d44bbebfab53a6cd9ede0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a925915099d44bbebfab53a6cd9ede0e" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 35605 } }
I20260809 09:37:15.535619 21346 raft_consensus.cc:388] T 00000000000000000000000000000000 P a925915099d44bbebfab53a6cd9ede0e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 09:37:15.535691 21346 raft_consensus.cc:482] T 00000000000000000000000000000000 P a925915099d44bbebfab53a6cd9ede0e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 09:37:15.535789 21346 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a925915099d44bbebfab53a6cd9ede0e [term 0 FOLLOWER]: Advancing to term 1
I20260809 09:37:15.536787 21346 raft_consensus.cc:504] T 00000000000000000000000000000000 P a925915099d44bbebfab53a6cd9ede0e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a925915099d44bbebfab53a6cd9ede0e" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 35605 } }
I20260809 09:37:15.537156 21346 leader_election.cc:302] T 00000000000000000000000000000000 P a925915099d44bbebfab53a6cd9ede0e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a925915099d44bbebfab53a6cd9ede0e; no voters:
I20260809 09:37:15.537479 21346 leader_election.cc:288] T 00000000000000000000000000000000 P a925915099d44bbebfab53a6cd9ede0e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 09:37:15.537592 21352 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a925915099d44bbebfab53a6cd9ede0e [term 1 FOLLOWER]: Leader election won for term 1
I20260809 09:37:15.537848 21352 raft_consensus.cc:686] T 00000000000000000000000000000000 P a925915099d44bbebfab53a6cd9ede0e [term 1 LEADER]: Becoming Leader. State: Replica: a925915099d44bbebfab53a6cd9ede0e, State: Running, Role: LEADER
I20260809 09:37:15.538232 21352 consensus_queue.cc:237] T 00000000000000000000000000000000 P a925915099d44bbebfab53a6cd9ede0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a925915099d44bbebfab53a6cd9ede0e" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 35605 } }
I20260809 09:37:15.538337 20982 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 09:37:15.538769 21346 sys_catalog.cc:564] T 00000000000000000000000000000000 P a925915099d44bbebfab53a6cd9ede0e [sys.catalog]: configured and running, proceeding with master startup.
I20260809 09:37:15.540027 21353 sys_catalog.cc:455] T 00000000000000000000000000000000 P a925915099d44bbebfab53a6cd9ede0e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a925915099d44bbebfab53a6cd9ede0e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a925915099d44bbebfab53a6cd9ede0e" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 35605 } } }
I20260809 09:37:15.540190 21353 sys_catalog.cc:458] T 00000000000000000000000000000000 P a925915099d44bbebfab53a6cd9ede0e [sys.catalog]: This master's current role is: LEADER
I20260809 09:37:15.540509 21356 sys_catalog.cc:455] T 00000000000000000000000000000000 P a925915099d44bbebfab53a6cd9ede0e [sys.catalog]: SysCatalogTable state changed. Reason: New leader a925915099d44bbebfab53a6cd9ede0e. Latest consensus state: current_term: 1 leader_uuid: "a925915099d44bbebfab53a6cd9ede0e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a925915099d44bbebfab53a6cd9ede0e" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 35605 } } }
I20260809 09:37:15.540603 21356 sys_catalog.cc:458] T 00000000000000000000000000000000 P a925915099d44bbebfab53a6cd9ede0e [sys.catalog]: This master's current role is: LEADER
I20260809 09:37:15.540618 21374 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 09:37:15.543542 21374 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 09:37:15.546504 21349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:37:15.541185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53353 (local address 127.20.125.190:35605)
0809 09:37:15.541298 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0809 09:37:15.541303 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 09:37:15.541694 (+ 391us) server_negotiation.cc:407] Connection header received
0809 09:37:15.542323 (+ 629us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:37:15.542326 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:37:15.542382 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:37:15.542492 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:37:15.542791 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:37:15.543548 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:37:15.544725 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:37:15.545062 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:37:15.545176 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:37:15.546052 (+ 876us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:37:15.546056 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:37:15.546059 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 09:37:15.546070 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:37:15.546103 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 09:37:15.546115 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:37:15.546198 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0809 09:37:15.546291 (+ 93us) server_negotiation.cc:299] Negotiation successful
0809 09:37:15.546340 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260809 09:37:15.550102 21374 catalog_manager.cc:1347] Generated new cluster ID: dc845dd12f574530bec2a4cee9aedb85
I20260809 09:37:15.550176 21374 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 09:37:15.561949 21374 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 09:37:15.563046 21374 catalog_manager.cc:1504] Loading token signing keys...
I20260809 09:37:15.572048 21374 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a925915099d44bbebfab53a6cd9ede0e: Generated new TSK 0
I20260809 09:37:15.572742 21374 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 09:37:15.580111 20982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu with pid 21256
2026-08-09T09:37:15Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1650 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786267941944493-20982-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 09:37:15 dist-test-slave-7kzw krb5kdc[21393](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 09:37:15 dist-test-slave-7kzw krb5kdc[21393](info): set up 2 sockets
Aug 09 09:37:15 dist-test-slave-7kzw krb5kdc[21393](info): commencing operation
krb5kdc: starting...
W20260809 09:37:16.895783 20982 mini_kdc.cc:121] Time spent starting KDC: real 1.290s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786267941944493-20982-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786267941944493-20982-0/krb5kdc/test-user.keytab.
Aug 09 09:37:16 dist-test-slave-7kzw krb5kdc[21393](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268236, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T09:37:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T09:37:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.125.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.125.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.125.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.190.keytab.
Entry for principal kudu/127.20.125.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.190.keytab.
I20260809 09:37:17.035797 20982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
/tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786267941944493-20982-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786267941944493-20982-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786267941944493-20982-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786267941944493-20982-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.125.190:35761
--webserver_interface=127.20.125.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.190.keytab
--principal=kudu/127.20.125.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.125.148:41525
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.125.190:35761
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786267941944493-20982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786267941944493-20982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786267941944493-20982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 09:37:17.151137 21412 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 09:37:17.151456 21412 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 09:37:17.151527 21412 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 09:37:17.155596 21412 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 09:37:17.155685 21412 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 09:37:17.155725 21412 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 09:37:17.155778 21412 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 09:37:17.155819 21412 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 09:37:17.161103 21412 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.125.148:41525
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786267941944493-20982-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786267941944493-20982-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.125.190:35761
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.190.keytab
--principal=kudu/127.20.125.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.125.190:35761
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786267941944493-20982-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.125.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21412
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786267941944493-20982-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 08:38:55 UTC on ccycloud.quasar-njjleu.root.comops.site
build id 4575935
I20260809 09:37:17.162212 21412 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 09:37:17.163271 21412 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 09:37:17.172194 21412 server_base.cc:1034] running on GCE node
W20260809 09:37:17.172098 21420 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:37:17.172165 21421 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:37:17.172116 21423 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:37:17.173005 21412 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 09:37:17.174212 21412 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 09:37:17.175421 21412 hybrid_clock.cc:648] HybridClock initialized: now 1786268237175392 us; error 41 us; skew 500 ppm
Aug 09 09:37:17 dist-test-slave-7kzw krb5kdc[21393](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268237, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 09:37:17.178344 21412 init.cc:377] Logged in from keytab as kudu/127.20.125.190@KRBTEST.COM (short username kudu)
I20260809 09:37:17.179328 21412 webserver.cc:460] Webserver started at http://127.20.125.190:36935/ using document root <none> and password file <none>
I20260809 09:37:17.179911 21412 fs_manager.cc:359] Metadata directory not provided
I20260809 09:37:17.180001 21412 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 09:37:17.180245 21412 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 09:37:17.182581 21412 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786267941944493-20982-0/minicluster-data/master-0/data/instance:
uuid: "c207a735234447eb8784d093c124007b"
format_stamp: "Formatted at 2026-08-09 09:37:17 on dist-test-slave-7kzw"
server_key: "50a4e094e48e6d9ddfd87ef7e5361f8a"
server_key_iv: "687dc075384a92e58c891c4a71b9da02"
server_key_version: "encryptionkey@0"
I20260809 09:37:17.183176 21412 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786267941944493-20982-0/minicluster-data/master-0/wal/instance:
uuid: "c207a735234447eb8784d093c124007b"
format_stamp: "Formatted at 2026-08-09 09:37:17 on dist-test-slave-7kzw"
server_key: "50a4e094e48e6d9ddfd87ef7e5361f8a"
server_key_iv: "687dc075384a92e58c891c4a71b9da02"
server_key_version: "encryptionkey@0"
I20260809 09:37:17.187011 21412 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260809 09:37:17.189482 21430 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 09:37:17.190764 21412 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 09:37:17.190907 21412 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786267941944493-20982-0/minicluster-data/master-0/data,/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786267941944493-20982-0/minicluster-data/master-0/wal
uuid: "c207a735234447eb8784d093c124007b"
format_stamp: "Formatted at 2026-08-09 09:37:17 on dist-test-slave-7kzw"
server_key: "50a4e094e48e6d9ddfd87ef7e5361f8a"
server_key_iv: "687dc075384a92e58c891c4a71b9da02"
server_key_version: "encryptionkey@0"
I20260809 09:37:17.191032 21412 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786267941944493-20982-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786267941944493-20982-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786267941944493-20982-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 09:37:17.195648 21412 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 09:37:17.199350 21412 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 09:37:17.199564 21412 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 09:37:17.207968 21412 rpc_server.cc:307] RPC server started. Bound to: 127.20.125.190:35761
I20260809 09:37:17.207999 21507 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.125.190:35761 every 8 connection(s)
I20260809 09:37:17.209055 21412 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786267941944493-20982-0/minicluster-data/master-0/data/info.pb
I20260809 09:37:17.212109 21509 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 09:37:17.213078 20982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu as pid 21412
I20260809 09:37:17.213248 20982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786267941944493-20982-0/minicluster-data/master-0/wal/instance
I20260809 09:37:17.213629 20982 external_mini_cluster.cc:1428] Setting key 7a8ecabecea447b7f5f254ddcf1c35a0
I20260809 09:37:17.219441 21509 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c207a735234447eb8784d093c124007b: Bootstrap starting.
I20260809 09:37:17.219738 21513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:37:17.215254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55175 (local address 127.20.125.190:35761)
0809 09:37:17.215769 (+ 515us) server_negotiation.cc:206] Beginning negotiation
0809 09:37:17.215790 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0809 09:37:17.215838 (+ 48us) server_negotiation.cc:407] Connection header received
0809 09:37:17.216738 (+ 900us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:37:17.216764 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:37:17.217138 (+ 374us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:37:17.217546 (+ 408us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:37:17.218244 (+ 698us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:37:17.218250 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:37:17.218266 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 09:37:17.218278 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:37:17.218348 (+ 70us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 09:37:17.218382 (+ 34us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:37:17.218571 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0809 09:37:17.218646 (+ 75us) server_negotiation.cc:299] Negotiation successful
0809 09:37:17.218800 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":328,"thread_start_us":139,"threads_started":1}
I20260809 09:37:17.222153 21509 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c207a735234447eb8784d093c124007b: Neither blocks nor log segments found. Creating new log.
I20260809 09:37:17.222930 20982 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 09:37:17.223176 21509 log.cc:824] T 00000000000000000000000000000000 P c207a735234447eb8784d093c124007b: Log is configured to *not* fsync() on all Append() calls
I20260809 09:37:17.225533 21509 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c207a735234447eb8784d093c124007b: No bootstrap required, opened a new log
I20260809 09:37:17.227556 21513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:37:17.225550 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38221 (local address 127.20.125.190:35761)
0809 09:37:17.225853 (+ 303us) server_negotiation.cc:206] Beginning negotiation
0809 09:37:17.225857 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 09:37:17.225979 (+ 122us) server_negotiation.cc:407] Connection header received
0809 09:37:17.226130 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:37:17.226133 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:37:17.226170 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:37:17.226249 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:37:17.227120 (+ 871us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:37:17.227124 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:37:17.227127 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 09:37:17.227141 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:37:17.227177 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 09:37:17.227189 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:37:17.227299 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0809 09:37:17.227382 (+ 83us) server_negotiation.cc:299] Negotiation successful
0809 09:37:17.227429 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":202}
I20260809 09:37:17.230089 21509 raft_consensus.cc:348] T 00000000000000000000000000000000 P c207a735234447eb8784d093c124007b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c207a735234447eb8784d093c124007b" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 35761 } }
I20260809 09:37:17.230341 21509 raft_consensus.cc:374] T 00000000000000000000000000000000 P c207a735234447eb8784d093c124007b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 09:37:17.230422 21509 raft_consensus.cc:729] T 00000000000000000000000000000000 P c207a735234447eb8784d093c124007b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c207a735234447eb8784d093c124007b, State: Initialized, Role: FOLLOWER
I20260809 09:37:17.230928 21509 consensus_queue.cc:260] T 00000000000000000000000000000000 P c207a735234447eb8784d093c124007b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c207a735234447eb8784d093c124007b" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 35761 } }
I20260809 09:37:17.231086 21509 raft_consensus.cc:388] T 00000000000000000000000000000000 P c207a735234447eb8784d093c124007b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 09:37:17.231158 21509 raft_consensus.cc:482] T 00000000000000000000000000000000 P c207a735234447eb8784d093c124007b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 09:37:17.231284 21509 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c207a735234447eb8784d093c124007b [term 0 FOLLOWER]: Advancing to term 1
I20260809 09:37:17.232267 21509 raft_consensus.cc:504] T 00000000000000000000000000000000 P c207a735234447eb8784d093c124007b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c207a735234447eb8784d093c124007b" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 35761 } }
I20260809 09:37:17.232626 21509 leader_election.cc:302] T 00000000000000000000000000000000 P c207a735234447eb8784d093c124007b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c207a735234447eb8784d093c124007b; no voters:
I20260809 09:37:17.232967 21509 leader_election.cc:288] T 00000000000000000000000000000000 P c207a735234447eb8784d093c124007b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 09:37:17.233119 21523 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c207a735234447eb8784d093c124007b [term 1 FOLLOWER]: Leader election won for term 1
I20260809 09:37:17.233386 21523 raft_consensus.cc:686] T 00000000000000000000000000000000 P c207a735234447eb8784d093c124007b [term 1 LEADER]: Becoming Leader. State: Replica: c207a735234447eb8784d093c124007b, State: Running, Role: LEADER
I20260809 09:37:17.233716 21523 consensus_queue.cc:237] T 00000000000000000000000000000000 P c207a735234447eb8784d093c124007b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c207a735234447eb8784d093c124007b" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 35761 } }
I20260809 09:37:17.234320 21509 sys_catalog.cc:564] T 00000000000000000000000000000000 P c207a735234447eb8784d093c124007b [sys.catalog]: configured and running, proceeding with master startup.
I20260809 09:37:17.234889 21523 sys_catalog.cc:455] T 00000000000000000000000000000000 P c207a735234447eb8784d093c124007b [sys.catalog]: SysCatalogTable state changed. Reason: New leader c207a735234447eb8784d093c124007b. Latest consensus state: current_term: 1 leader_uuid: "c207a735234447eb8784d093c124007b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c207a735234447eb8784d093c124007b" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 35761 } } }
I20260809 09:37:17.235019 21523 sys_catalog.cc:458] T 00000000000000000000000000000000 P c207a735234447eb8784d093c124007b [sys.catalog]: This master's current role is: LEADER
I20260809 09:37:17.235498 21530 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 09:37:17.235471 21527 sys_catalog.cc:455] T 00000000000000000000000000000000 P c207a735234447eb8784d093c124007b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c207a735234447eb8784d093c124007b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c207a735234447eb8784d093c124007b" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 35761 } } }
I20260809 09:37:17.235661 21527 sys_catalog.cc:458] T 00000000000000000000000000000000 P c207a735234447eb8784d093c124007b [sys.catalog]: This master's current role is: LEADER
I20260809 09:37:17.238447 21530 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 09:37:17.244349 21530 catalog_manager.cc:1347] Generated new cluster ID: 24c9836637804ff198d4151eb5f3a7b1
I20260809 09:37:17.244417 21530 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 09:37:17.256834 21530 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 09:37:17.257155 21530 catalog_manager.cc:1504] Loading token signing keys...
I20260809 09:37:17.273347 21530 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c207a735234447eb8784d093c124007b: Generated new TSK 0
I20260809 09:37:17.274231 21530 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 09:37:17.286259 20982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu with pid 21412
2026-08-09T09:37:17Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1718 ms)
[----------] 18 tests from AuthTokenIssuingTest (30015 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786267941944493-20982-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 09:37:17 dist-test-slave-7kzw krb5kdc[21558](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 09:37:17 dist-test-slave-7kzw krb5kdc[21558](info): set up 2 sockets
Aug 09 09:37:17 dist-test-slave-7kzw krb5kdc[21558](info): commencing operation
krb5kdc: starting...
W20260809 09:37:18.577225 20982 mini_kdc.cc:121] Time spent starting KDC: real 1.252s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786267941944493-20982-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786267941944493-20982-0/krb5kdc/test-user.keytab.
Aug 09 09:37:18 dist-test-slave-7kzw krb5kdc[21558](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268238, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T09:37:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T09:37:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.125.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.125.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.125.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.190.keytab.
Entry for principal kudu/127.20.125.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.190.keytab.
I20260809 09:37:18.729776 20982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
/tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786267941944493-20982-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786267941944493-20982-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786267941944493-20982-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786267941944493-20982-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.125.190:35987
--webserver_interface=127.20.125.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.190.keytab
--principal=kudu/127.20.125.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.125.148:44631
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.125.190:35987,127.20.125.189:44001,127.20.125.188:43753
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786267941944493-20982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786267941944493-20982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786267941944493-20982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 09:37:18.846203 21576 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 09:37:18.846585 21576 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 09:37:18.846626 21576 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 09:37:18.846665 21576 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 09:37:18.850663 21576 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 09:37:18.850721 21576 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 09:37:18.850740 21576 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 09:37:18.850757 21576 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 09:37:18.850772 21576 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 09:37:18.855998 21576 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.125.148:44631
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786267941944493-20982-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786267941944493-20982-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.125.190:35987,127.20.125.189:44001,127.20.125.188:43753
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.190.keytab
--principal=kudu/127.20.125.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.125.190:35987
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786267941944493-20982-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.125.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21576
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786267941944493-20982-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 08:38:55 UTC on ccycloud.quasar-njjleu.root.comops.site
build id 4575935
I20260809 09:37:18.857102 21576 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 09:37:18.858108 21576 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 09:37:18.866760 21591 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:37:18.866657 21596 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:37:18.866683 21592 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:37:18.867012 21576 server_base.cc:1034] running on GCE node
I20260809 09:37:18.867633 21576 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 09:37:18.869060 21576 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 09:37:18.870265 21576 hybrid_clock.cc:648] HybridClock initialized: now 1786268238870239 us; error 45 us; skew 500 ppm
Aug 09 09:37:18 dist-test-slave-7kzw krb5kdc[21558](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268238, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 09:37:18.874270 21576 init.cc:377] Logged in from keytab as kudu/127.20.125.190@KRBTEST.COM (short username kudu)
I20260809 09:37:18.875618 21576 webserver.cc:460] Webserver started at http://127.20.125.190:35889/ using document root <none> and password file <none>
I20260809 09:37:18.876268 21576 fs_manager.cc:359] Metadata directory not provided
I20260809 09:37:18.876348 21576 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 09:37:18.876571 21576 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 09:37:18.879544 21576 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786267941944493-20982-0/minicluster-data/master-0/data/instance:
uuid: "b673fd5cc908430ca7884785433738ce"
format_stamp: "Formatted at 2026-08-09 09:37:18 on dist-test-slave-7kzw"
server_key: "d36f4b1f1fd8d960d4a7a5154db651d1"
server_key_iv: "148b5ae3eb9d5884340173a64bae3cae"
server_key_version: "encryptionkey@0"
I20260809 09:37:18.880339 21576 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786267941944493-20982-0/minicluster-data/master-0/wal/instance:
uuid: "b673fd5cc908430ca7884785433738ce"
format_stamp: "Formatted at 2026-08-09 09:37:18 on dist-test-slave-7kzw"
server_key: "d36f4b1f1fd8d960d4a7a5154db651d1"
server_key_iv: "148b5ae3eb9d5884340173a64bae3cae"
server_key_version: "encryptionkey@0"
I20260809 09:37:18.884347 21576 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.003s
I20260809 09:37:18.886945 21605 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 09:37:18.888399 21576 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 09:37:18.888530 21576 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786267941944493-20982-0/minicluster-data/master-0/data,/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786267941944493-20982-0/minicluster-data/master-0/wal
uuid: "b673fd5cc908430ca7884785433738ce"
format_stamp: "Formatted at 2026-08-09 09:37:18 on dist-test-slave-7kzw"
server_key: "d36f4b1f1fd8d960d4a7a5154db651d1"
server_key_iv: "148b5ae3eb9d5884340173a64bae3cae"
server_key_version: "encryptionkey@0"
I20260809 09:37:18.888633 21576 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786267941944493-20982-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786267941944493-20982-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786267941944493-20982-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 09:37:18.916313 21576 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 09:37:18.919772 21576 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 09:37:18.919942 21576 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 09:37:18.927284 21576 rpc_server.cc:307] RPC server started. Bound to: 127.20.125.190:35987
I20260809 09:37:18.927626 21681 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.125.190:35987 every 8 connection(s)
I20260809 09:37:18.928529 21576 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786267941944493-20982-0/minicluster-data/master-0/data/info.pb
I20260809 09:37:18.931771 21683 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 09:37:18.935854 21683 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.125.190" port: 35987 } has no permanent_uuid. Determining permanent_uuid...
I20260809 09:37:18.938786 20982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu as pid 21576
I20260809 09:37:18.938951 20982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786267941944493-20982-0/minicluster-data/master-0/wal/instance
I20260809 09:37:18.939303 20982 external_mini_cluster.cc:1428] Setting key f945613535f2f34afe8d8f3f679c7bfb
Aug 09 09:37:18 dist-test-slave-7kzw krb5kdc[21558](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268238, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.190@KRBTEST.COM for kudu/127.20.125.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.20.125.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.125.189@KRBTEST.COM" created.
I20260809 09:37:18.953806 21688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:37:18.938204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41984 (local address 127.20.125.190:35987)
0809 09:37:18.939264 (+ 1060us) server_negotiation.cc:206] Beginning negotiation
0809 09:37:18.939283 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0809 09:37:18.939534 (+ 251us) server_negotiation.cc:407] Connection header received
0809 09:37:18.940639 (+ 1105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:37:18.940648 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:37:18.940887 (+ 239us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:37:18.941130 (+ 243us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:37:18.942267 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:37:18.943333 (+ 1066us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:37:18.944935 (+ 1602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:37:18.945361 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:37:18.945492 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:37:18.948755 (+ 3263us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:37:18.948765 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:37:18.948783 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:37:18.948814 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:37:18.950670 (+ 1856us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:37:18.951150 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:37:18.951155 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:37:18.951157 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:37:18.951265 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:37:18.951578 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:37:18.951581 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:37:18.951582 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:37:18.951968 (+ 386us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:37:18.952081 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0809 09:37:18.952720 (+ 639us) server_negotiation.cc:299] Negotiation successful
0809 09:37:18.953026 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":116,"server-negotiator.queue_time_us":899,"thread_start_us":142,"threads_started":1}
I20260809 09:37:18.953806 21689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:37:18.938205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:35987 (local address 127.0.0.1:41984)
0809 09:37:18.939340 (+ 1135us) negotiation.cc:107] Waiting for socket to connect
0809 09:37:18.939385 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 09:37:18.940305 (+ 920us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:37:18.941147 (+ 842us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:37:18.941152 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:37:18.941661 (+ 509us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:37:18.942064 (+ 403us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:37:18.942084 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:37:18.943478 (+ 1394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:37:18.943485 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:37:18.944791 (+ 1306us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:37:18.944805 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:37:18.945539 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:37:18.945544 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:37:18.945707 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:37:18.946413 (+ 706us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 09:37:18.946441 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:37:18.948575 (+ 2134us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 09:37:18.950775 (+ 2200us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:37:18.950784 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:37:18.950792 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:37:18.951031 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:37:18.951353 (+ 322us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:37:18.951355 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:37:18.951356 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:37:18.951472 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:37:18.952211 (+ 739us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 09:37:18.952222 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 09:37:18.952373 (+ 151us) client_negotiation.cc:769] Sending connection context
0809 09:37:18.952574 (+ 201us) client_negotiation.cc:240] Negotiation successful
0809 09:37:18.952946 (+ 372us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":945,"spinlock_wait_cycles":17792,"thread_start_us":222,"threads_started":1}
I20260809 09:37:18.956436 21683 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.125.189" port: 44001 } has no permanent_uuid. Determining permanent_uuid...
I20260809 09:37:18.957587 21689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:37:18.957195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.189:44001 (local address 127.0.0.1:49030)
0809 09:37:18.957377 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0809 09:37:18.957485 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.189:44001: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260809 09:37:18.957830 21609 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.20.125.189:44001: connect: Connection refused (error 111)
I20260809 09:37:18.958863 21689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:37:18.958604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.189:44001 (local address 127.0.0.1:49038)
0809 09:37:18.958762 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0809 09:37:18.958811 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.189:44001: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
W20260809 09:37:18.959026 21683 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.125.189:44001: Network error: Client connection negotiation failed: client connection to 127.20.125.189:44001: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.125.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.189.keytab.
Entry for principal kudu/127.20.125.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.189.keytab.
I20260809 09:37:18.969233 20982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
/tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786267941944493-20982-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786267941944493-20982-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786267941944493-20982-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786267941944493-20982-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.125.189:44001
--webserver_interface=127.20.125.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.189.keytab
--principal=kudu/127.20.125.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.125.148:44631
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.125.190:35987,127.20.125.189:44001,127.20.125.188:43753
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786267941944493-20982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786267941944493-20982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786267941944493-20982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 09:37:19.008285 21683 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.125.189" port: 44001 } attempt: 1
I20260809 09:37:19.009285 21689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:37:19.008937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.189:44001 (local address 127.0.0.1:49048)
0809 09:37:19.009098 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0809 09:37:19.009196 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.189:44001: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260809 09:37:19.010183 21689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:37:19.009954 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.189:44001 (local address 127.0.0.1:49054)
0809 09:37:19.010066 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0809 09:37:19.010117 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.189:44001: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260809 09:37:19.010339 21683 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.125.189:44001: Network error: Client connection negotiation failed: client connection to 127.20.125.189:44001: connect: Connection refused (error 111)
I20260809 09:37:19.078574 21683 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.125.189" port: 44001 } attempt: 2
I20260809 09:37:19.079552 21689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:37:19.079230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.189:44001 (local address 127.0.0.1:49060)
0809 09:37:19.079399 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0809 09:37:19.079468 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.189:44001: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260809 09:37:19.080451 21689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:37:19.080209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.189:44001 (local address 127.0.0.1:49070)
0809 09:37:19.080340 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0809 09:37:19.080392 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.189:44001: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
W20260809 09:37:19.080617 21683 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.125.189:44001: Network error: Client connection negotiation failed: client connection to 127.20.125.189:44001: connect: Connection refused (error 111)
W20260809 09:37:19.089929 21696 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 09:37:19.090216 21696 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 09:37:19.090270 21696 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 09:37:19.090358 21696 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 09:37:19.094651 21696 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 09:37:19.094714 21696 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 09:37:19.094735 21696 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 09:37:19.094753 21696 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 09:37:19.094767 21696 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 09:37:19.100272 21696 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.125.148:44631
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786267941944493-20982-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786267941944493-20982-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.20.125.190:35987,127.20.125.189:44001,127.20.125.188:43753
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.189.keytab
--principal=kudu/127.20.125.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.125.189:44001
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786267941944493-20982-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.20.125.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21696
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786267941944493-20982-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 08:38:55 UTC on ccycloud.quasar-njjleu.root.comops.site
build id 4575935
I20260809 09:37:19.101431 21696 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 09:37:19.102536 21696 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 09:37:19.111500 21702 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:37:19.111529 21704 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:37:19.111550 21701 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:37:19.111855 21696 server_base.cc:1034] running on GCE node
I20260809 09:37:19.112552 21696 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 09:37:19.113741 21696 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 09:37:19.114953 21696 hybrid_clock.cc:648] HybridClock initialized: now 1786268239114926 us; error 36 us; skew 500 ppm
Aug 09 09:37:19 dist-test-slave-7kzw krb5kdc[21558](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268239, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 09:37:19.118225 21696 init.cc:377] Logged in from keytab as kudu/127.20.125.189@KRBTEST.COM (short username kudu)
I20260809 09:37:19.119243 21696 webserver.cc:460] Webserver started at http://127.20.125.189:35363/ using document root <none> and password file <none>
I20260809 09:37:19.119819 21696 fs_manager.cc:359] Metadata directory not provided
I20260809 09:37:19.119889 21696 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 09:37:19.120069 21696 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 09:37:19.122291 21696 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786267941944493-20982-0/minicluster-data/master-1/data/instance:
uuid: "151817845efc4227a6d3735174ea5b0e"
format_stamp: "Formatted at 2026-08-09 09:37:19 on dist-test-slave-7kzw"
server_key: "b66c01421fb01e92c3b58b02e8b6a2e8"
server_key_iv: "88cc6ba1bbe293d33bc8ec704d937956"
server_key_version: "encryptionkey@0"
I20260809 09:37:19.122941 21696 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786267941944493-20982-0/minicluster-data/master-1/wal/instance:
uuid: "151817845efc4227a6d3735174ea5b0e"
format_stamp: "Formatted at 2026-08-09 09:37:19 on dist-test-slave-7kzw"
server_key: "b66c01421fb01e92c3b58b02e8b6a2e8"
server_key_iv: "88cc6ba1bbe293d33bc8ec704d937956"
server_key_version: "encryptionkey@0"
I20260809 09:37:19.127193 21696 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260809 09:37:19.129956 21716 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 09:37:19.131613 21696 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 09:37:19.131789 21696 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786267941944493-20982-0/minicluster-data/master-1/data,/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786267941944493-20982-0/minicluster-data/master-1/wal
uuid: "151817845efc4227a6d3735174ea5b0e"
format_stamp: "Formatted at 2026-08-09 09:37:19 on dist-test-slave-7kzw"
server_key: "b66c01421fb01e92c3b58b02e8b6a2e8"
server_key_iv: "88cc6ba1bbe293d33bc8ec704d937956"
server_key_version: "encryptionkey@0"
I20260809 09:37:19.131898 21696 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786267941944493-20982-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786267941944493-20982-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786267941944493-20982-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 09:37:19.156862 21696 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 09:37:19.160476 21696 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 09:37:19.160720 21696 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 09:37:19.168713 21696 rpc_server.cc:307] RPC server started. Bound to: 127.20.125.189:44001
I20260809 09:37:19.169049 21792 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.125.189:44001 every 8 connection(s)
I20260809 09:37:19.169795 21696 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786267941944493-20982-0/minicluster-data/master-1/data/info.pb
I20260809 09:37:19.171847 21683 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.125.189" port: 44001 } attempt: 3
I20260809 09:37:19.173092 21793 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 09:37:19.177351 20982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu as pid 21696
I20260809 09:37:19.177521 20982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786267941944493-20982-0/minicluster-data/master-1/wal/instance
I20260809 09:37:19.177848 20982 external_mini_cluster.cc:1428] Setting key 9c462b68359a34b8e99fa128c29c88c2
I20260809 09:37:19.177695 21793 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.125.190" port: 35987 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 09:37:19 dist-test-slave-7kzw krb5kdc[21558](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268238, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.190@KRBTEST.COM for kudu/127.20.125.189@KRBTEST.COM
I20260809 09:37:19.186790 21689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:37:19.172473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.189:44001 (local address 127.0.0.1:49074)
0809 09:37:19.172645 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0809 09:37:19.172665 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 09:37:19.172788 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:37:19.175245 (+ 2457us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:37:19.175249 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:37:19.175473 (+ 224us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:37:19.175620 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:37:19.175632 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:37:19.176971 (+ 1339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:37:19.176975 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:37:19.177907 (+ 932us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:37:19.177920 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:37:19.179153 (+ 1233us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:37:19.179158 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:37:19.179290 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:37:19.179936 (+ 646us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 09:37:19.179952 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:37:19.181802 (+ 1850us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 09:37:19.184492 (+ 2690us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:37:19.184499 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:37:19.184505 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:37:19.184872 (+ 367us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:37:19.185347 (+ 475us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:37:19.185351 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:37:19.185355 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:37:19.185473 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:37:19.186144 (+ 671us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 09:37:19.186147 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 09:37:19.186258 (+ 111us) client_negotiation.cc:769] Sending connection context
0809 09:37:19.186378 (+ 120us) client_negotiation.cc:240] Negotiation successful
0809 09:37:19.186571 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":59}
Aug 09 09:37:19 dist-test-slave-7kzw krb5kdc[21558](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268239, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.189@KRBTEST.COM for kudu/127.20.125.190@KRBTEST.COM
I20260809 09:37:19.188150 21800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:37:19.173282 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49074 (local address 127.20.125.189:44001)
0809 09:37:19.173791 (+ 509us) server_negotiation.cc:206] Beginning negotiation
0809 09:37:19.173806 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 09:37:19.173844 (+ 38us) server_negotiation.cc:407] Connection header received
0809 09:37:19.174609 (+ 765us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:37:19.174629 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:37:19.174954 (+ 325us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:37:19.175238 (+ 284us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:37:19.175800 (+ 562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:37:19.176828 (+ 1028us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:37:19.178050 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:37:19.178977 (+ 927us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:37:19.179178 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:37:19.182030 (+ 2852us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:37:19.182040 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:37:19.182052 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:37:19.182096 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:37:19.184304 (+ 2208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:37:19.185041 (+ 737us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:37:19.185047 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:37:19.185055 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:37:19.185231 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:37:19.185589 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:37:19.185593 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:37:19.185595 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:37:19.186016 (+ 421us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:37:19.186162 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0809 09:37:19.186570 (+ 408us) server_negotiation.cc:299] Negotiation successful
0809 09:37:19.187201 (+ 631us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":354,"thread_start_us":146,"threads_started":1}
WARNING: no policy specified for kudu/127.20.125.188@KRBTEST.COM; defaulting to no policy
I20260809 09:37:19.190869 21683 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.125.188" port: 43753 } has no permanent_uuid. Determining permanent_uuid...
I20260809 09:37:19.191769 21689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:37:19.191450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.188:43753 (local address 127.0.0.1:50808)
0809 09:37:19.191555 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0809 09:37:19.191619 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.188:43753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260809 09:37:19.192106 21688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:37:19.179693 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41990 (local address 127.20.125.190:35987)
0809 09:37:19.179845 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0809 09:37:19.179854 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 09:37:19.180340 (+ 486us) server_negotiation.cc:407] Connection header received
0809 09:37:19.180586 (+ 246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:37:19.180591 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:37:19.180648 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:37:19.180749 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:37:19.181538 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:37:19.182874 (+ 1336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:37:19.184433 (+ 1559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:37:19.184778 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:37:19.184911 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:37:19.188015 (+ 3104us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:37:19.188019 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:37:19.188022 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:37:19.188050 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:37:19.189854 (+ 1804us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:37:19.190513 (+ 659us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:37:19.190517 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:37:19.190519 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:37:19.190625 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:37:19.191039 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:37:19.191043 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:37:19.191045 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:37:19.191294 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:37:19.191409 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0809 09:37:19.191771 (+ 362us) server_negotiation.cc:299] Negotiation successful
0809 09:37:19.191941 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 09:37:19.192159 21802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:37:19.179678 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:35987 (local address 127.0.0.1:41990)
0809 09:37:19.180183 (+ 505us) negotiation.cc:107] Waiting for socket to connect
0809 09:37:19.180253 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0809 09:37:19.180445 (+ 192us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:37:19.180819 (+ 374us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:37:19.180828 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:37:19.181201 (+ 373us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:37:19.181335 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:37:19.181341 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:37:19.183039 (+ 1698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:37:19.183047 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:37:19.184267 (+ 1220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:37:19.184284 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:37:19.184913 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:37:19.184917 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:37:19.185059 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:37:19.185600 (+ 541us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 09:37:19.185668 (+ 68us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:37:19.187850 (+ 2182us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 09:37:19.189981 (+ 2131us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:37:19.189991 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:37:19.189997 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:37:19.190337 (+ 340us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:37:19.190779 (+ 442us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:37:19.190785 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:37:19.190787 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:37:19.190895 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:37:19.191417 (+ 522us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 09:37:19.191423 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 09:37:19.191531 (+ 108us) client_negotiation.cc:769] Sending connection context
0809 09:37:19.191697 (+ 166us) client_negotiation.cc:240] Negotiation successful
0809 09:37:19.191952 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":290,"spinlock_wait_cycles":768,"thread_start_us":179,"threads_started":1}
I20260809 09:37:19.193163 21689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:37:19.192799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.188:43753 (local address 127.0.0.1:50820)
0809 09:37:19.193030 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0809 09:37:19.193083 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.188:43753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":164}
W20260809 09:37:19.193360 21683 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.125.188:43753: Network error: Client connection negotiation failed: client connection to 127.20.125.188:43753: connect: Connection refused (error 111)
I20260809 09:37:19.193432 21793 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.125.189" port: 44001 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.125.188@KRBTEST.COM" created.
Aug 09 09:37:19 dist-test-slave-7kzw krb5kdc[21558](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268239, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.189@KRBTEST.COM for kudu/127.20.125.189@KRBTEST.COM
I20260809 09:37:19.203336 21802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:37:19.194029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.189:44001 (local address 127.0.0.1:49078)
0809 09:37:19.194265 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0809 09:37:19.194281 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 09:37:19.194399 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:37:19.194723 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:37:19.194729 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:37:19.194947 (+ 218us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:37:19.195139 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:37:19.195151 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:37:19.195991 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:37:19.195998 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:37:19.196744 (+ 746us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:37:19.196754 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:37:19.197451 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:37:19.197454 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:37:19.197535 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:37:19.198065 (+ 530us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 09:37:19.198086 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:37:19.199815 (+ 1729us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 09:37:19.201700 (+ 1885us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:37:19.201707 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:37:19.201710 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:37:19.201918 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:37:19.202209 (+ 291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:37:19.202214 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:37:19.202216 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:37:19.202306 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:37:19.202766 (+ 460us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 09:37:19.202769 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 09:37:19.202869 (+ 100us) client_negotiation.cc:769] Sending connection context
0809 09:37:19.202979 (+ 110us) client_negotiation.cc:240] Negotiation successful
0809 09:37:19.203138 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159}
I20260809 09:37:19.203339 21800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:37:19.194126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49078 (local address 127.20.125.189:44001)
0809 09:37:19.194243 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0809 09:37:19.194247 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 09:37:19.194345 (+ 98us) server_negotiation.cc:407] Connection header received
0809 09:37:19.194554 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:37:19.194557 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:37:19.194603 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:37:19.194676 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:37:19.195292 (+ 616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:37:19.195856 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:37:19.196882 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:37:19.197304 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:37:19.197428 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:37:19.199985 (+ 2557us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:37:19.199988 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:37:19.199990 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:37:19.200012 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:37:19.201573 (+ 1561us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:37:19.202020 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:37:19.202025 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:37:19.202027 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:37:19.202107 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:37:19.202406 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:37:19.202409 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:37:19.202411 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:37:19.202662 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:37:19.202778 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0809 09:37:19.203040 (+ 262us) server_negotiation.cc:299] Negotiation successful
0809 09:37:19.203210 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":51,"server-negotiator.queue_time_us":41}
I20260809 09:37:19.204427 21793 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.125.188" port: 43753 } has no permanent_uuid. Determining permanent_uuid...
I20260809 09:37:19.205152 21802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:37:19.204871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.188:43753 (local address 127.0.0.1:50826)
0809 09:37:19.204993 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0809 09:37:19.205091 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.188:43753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260809 09:37:19.205734 21717 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.20.125.188:43753: connect: Connection refused (error 111)
I20260809 09:37:19.206722 21802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:37:19.206455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.188:43753 (local address 127.0.0.1:50832)
0809 09:37:19.206603 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0809 09:37:19.206666 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.188:43753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
W20260809 09:37:19.206903 21793 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.125.188:43753: Network error: Client connection negotiation failed: client connection to 127.20.125.188:43753: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.125.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.188.keytab.
Entry for principal kudu/127.20.125.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.188.keytab.
I20260809 09:37:19.214668 20982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
/tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786267941944493-20982-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786267941944493-20982-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786267941944493-20982-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786267941944493-20982-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.125.188:43753
--webserver_interface=127.20.125.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.188.keytab
--principal=kudu/127.20.125.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.125.148:44631
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.125.190:35987,127.20.125.189:44001,127.20.125.188:43753
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786267941944493-20982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786267941944493-20982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786267941944493-20982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 09:37:19.224536 21683 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.125.188" port: 43753 } attempt: 1
I20260809 09:37:19.225534 21689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:37:19.225113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.188:43753 (local address 127.0.0.1:50842)
0809 09:37:19.225327 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0809 09:37:19.225416 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.188:43753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260809 09:37:19.226410 21689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:37:19.226170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.188:43753 (local address 127.0.0.1:50856)
0809 09:37:19.226286 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0809 09:37:19.226342 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.188:43753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260809 09:37:19.226581 21683 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.125.188:43753: Network error: Client connection negotiation failed: client connection to 127.20.125.188:43753: connect: Connection refused (error 111)
I20260809 09:37:19.256160 21793 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.125.188" port: 43753 } attempt: 1
I20260809 09:37:19.257223 21802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:37:19.256902 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.188:43753 (local address 127.0.0.1:50870)
0809 09:37:19.257063 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0809 09:37:19.257130 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.188:43753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260809 09:37:19.258018 21802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:37:19.257774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.188:43753 (local address 127.0.0.1:50882)
0809 09:37:19.257904 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0809 09:37:19.257955 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.188:43753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260809 09:37:19.258323 21793 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.125.188:43753: Network error: Client connection negotiation failed: client connection to 127.20.125.188:43753: connect: Connection refused (error 111)
I20260809 09:37:19.301753 21683 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.125.188" port: 43753 } attempt: 2
I20260809 09:37:19.302712 21689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:37:19.302321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.188:43753 (local address 127.0.0.1:50884)
0809 09:37:19.302527 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0809 09:37:19.302612 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.188:43753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260809 09:37:19.303532 21689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:37:19.303281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.188:43753 (local address 127.0.0.1:50886)
0809 09:37:19.303407 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0809 09:37:19.303465 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.188:43753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260809 09:37:19.303680 21683 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.125.188:43753: Network error: Client connection negotiation failed: client connection to 127.20.125.188:43753: connect: Connection refused (error 111)
I20260809 09:37:19.326557 21793 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.125.188" port: 43753 } attempt: 2
I20260809 09:37:19.327477 21802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:37:19.327147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.188:43753 (local address 127.0.0.1:50898)
0809 09:37:19.327318 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0809 09:37:19.327393 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.188:43753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260809 09:37:19.328482 21802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:37:19.328184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.188:43753 (local address 127.0.0.1:50910)
0809 09:37:19.328339 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0809 09:37:19.328402 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.188:43753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
W20260809 09:37:19.328642 21793 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.125.188:43753: Network error: Client connection negotiation failed: client connection to 127.20.125.188:43753: connect: Connection refused (error 111)
W20260809 09:37:19.333945 21806 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 09:37:19.334311 21806 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 09:37:19.334385 21806 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 09:37:19.334529 21806 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 09:37:19.339295 21806 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 09:37:19.339385 21806 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 09:37:19.339442 21806 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 09:37:19.339490 21806 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 09:37:19.339545 21806 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 09:37:19.345320 21806 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.125.148:44631
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786267941944493-20982-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786267941944493-20982-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.20.125.190:35987,127.20.125.189:44001,127.20.125.188:43753
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.188.keytab
--principal=kudu/127.20.125.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.125.188:43753
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786267941944493-20982-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.20.125.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21806
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786267941944493-20982-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 08:38:55 UTC on ccycloud.quasar-njjleu.root.comops.site
build id 4575935
I20260809 09:37:19.346405 21806 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 09:37:19.347508 21806 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 09:37:19.356432 21815 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:37:19.356359 21817 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:37:19.356375 21814 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:37:19.356727 21806 server_base.cc:1034] running on GCE node
I20260809 09:37:19.357318 21806 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 09:37:19.358438 21806 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 09:37:19.359684 21806 hybrid_clock.cc:648] HybridClock initialized: now 1786268239359663 us; error 28 us; skew 500 ppm
Aug 09 09:37:19 dist-test-slave-7kzw krb5kdc[21558](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268239, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 09:37:19.362715 21806 init.cc:377] Logged in from keytab as kudu/127.20.125.188@KRBTEST.COM (short username kudu)
I20260809 09:37:19.363596 21806 webserver.cc:460] Webserver started at http://127.20.125.188:45305/ using document root <none> and password file <none>
I20260809 09:37:19.364215 21806 fs_manager.cc:359] Metadata directory not provided
I20260809 09:37:19.364312 21806 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 09:37:19.364539 21806 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 09:37:19.366546 21806 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786267941944493-20982-0/minicluster-data/master-2/data/instance:
uuid: "3b07b93bacc34ee69578538bc8afa102"
format_stamp: "Formatted at 2026-08-09 09:37:19 on dist-test-slave-7kzw"
server_key: "04d726d277ab95ac2f76f4579800150f"
server_key_iv: "40dc9ab56e42ed52d618a66140512d12"
server_key_version: "encryptionkey@0"
I20260809 09:37:19.367102 21806 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786267941944493-20982-0/minicluster-data/master-2/wal/instance:
uuid: "3b07b93bacc34ee69578538bc8afa102"
format_stamp: "Formatted at 2026-08-09 09:37:19 on dist-test-slave-7kzw"
server_key: "04d726d277ab95ac2f76f4579800150f"
server_key_iv: "40dc9ab56e42ed52d618a66140512d12"
server_key_version: "encryptionkey@0"
I20260809 09:37:19.370757 21806 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.002s
I20260809 09:37:19.373301 21828 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 09:37:19.374694 21806 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 09:37:19.374854 21806 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786267941944493-20982-0/minicluster-data/master-2/data,/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786267941944493-20982-0/minicluster-data/master-2/wal
uuid: "3b07b93bacc34ee69578538bc8afa102"
format_stamp: "Formatted at 2026-08-09 09:37:19 on dist-test-slave-7kzw"
server_key: "04d726d277ab95ac2f76f4579800150f"
server_key_iv: "40dc9ab56e42ed52d618a66140512d12"
server_key_version: "encryptionkey@0"
I20260809 09:37:19.374970 21806 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786267941944493-20982-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786267941944493-20982-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786267941944493-20982-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 09:37:19.402839 21683 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.125.188" port: 43753 } attempt: 3
I20260809 09:37:19.403700 21689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:37:19.403391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.188:43753 (local address 127.0.0.1:50926)
0809 09:37:19.403541 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0809 09:37:19.403612 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.188:43753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260809 09:37:19.404508 21689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:37:19.404269 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.188:43753 (local address 127.0.0.1:50942)
0809 09:37:19.404390 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0809 09:37:19.404445 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.188:43753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260809 09:37:19.404676 21683 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.125.188:43753: Network error: Client connection negotiation failed: client connection to 127.20.125.188:43753: connect: Connection refused (error 111)
I20260809 09:37:19.419847 21793 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.125.188" port: 43753 } attempt: 3
I20260809 09:37:19.420722 21802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:37:19.420393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.188:43753 (local address 127.0.0.1:50958)
0809 09:37:19.420547 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0809 09:37:19.420643 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.188:43753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260809 09:37:19.421597 21802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:37:19.421276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.188:43753 (local address 127.0.0.1:50968)
0809 09:37:19.421480 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0809 09:37:19.421528 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.188:43753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121}
W20260809 09:37:19.421749 21793 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.125.188:43753: Network error: Client connection negotiation failed: client connection to 127.20.125.188:43753: connect: Connection refused (error 111)
I20260809 09:37:19.426285 21806 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 09:37:19.429890 21806 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 09:37:19.430105 21806 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 09:37:19.438050 21806 rpc_server.cc:307] RPC server started. Bound to: 127.20.125.188:43753
I20260809 09:37:19.438407 21908 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.125.188:43753 every 8 connection(s)
I20260809 09:37:19.439138 21806 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786267941944493-20982-0/minicluster-data/master-2/data/info.pb
I20260809 09:37:19.442124 21909 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 09:37:19.442529 20982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu as pid 21806
I20260809 09:37:19.442693 20982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786267941944493-20982-0/minicluster-data/master-2/wal/instance
I20260809 09:37:19.443006 20982 external_mini_cluster.cc:1428] Setting key 2efd0cf85d81bf86055cde7db22a3f25
I20260809 09:37:19.447708 21909 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.125.190" port: 35987 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 09:37:19 dist-test-slave-7kzw krb5kdc[21558](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268238, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.125.188@KRBTEST.COM
Aug 09 09:37:19 dist-test-slave-7kzw krb5kdc[21558](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268239, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.188@KRBTEST.COM for kudu/127.20.125.190@KRBTEST.COM
I20260809 09:37:19.460289 21914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:37:19.444382 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50970 (local address 127.20.125.188:43753)
0809 09:37:19.444825 (+ 443us) server_negotiation.cc:206] Beginning negotiation
0809 09:37:19.444848 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0809 09:37:19.444894 (+ 46us) server_negotiation.cc:407] Connection header received
0809 09:37:19.445864 (+ 970us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:37:19.445891 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:37:19.446346 (+ 455us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:37:19.446776 (+ 430us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:37:19.447612 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:37:19.449247 (+ 1635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:37:19.450626 (+ 1379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:37:19.451218 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:37:19.451347 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:37:19.454712 (+ 3365us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:37:19.454718 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:37:19.454728 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:37:19.454778 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:37:19.456727 (+ 1949us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:37:19.457251 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:37:19.457262 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:37:19.457270 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:37:19.457443 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:37:19.457789 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:37:19.457792 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:37:19.457793 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:37:19.458362 (+ 569us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:37:19.458630 (+ 268us) server_negotiation.cc:1035] Waiting for connection context
0809 09:37:19.458880 (+ 250us) server_negotiation.cc:299] Negotiation successful
0809 09:37:19.459203 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":285,"thread_start_us":106,"threads_started":1}
I20260809 09:37:19.460645 21688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:37:19.449858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41998 (local address 127.20.125.190:35987)
0809 09:37:19.450011 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0809 09:37:19.450016 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 09:37:19.450339 (+ 323us) server_negotiation.cc:407] Connection header received
0809 09:37:19.450535 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:37:19.450542 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:37:19.450623 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:37:19.450717 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:37:19.451493 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:37:19.452014 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:37:19.453491 (+ 1477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:37:19.453893 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:37:19.454018 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:37:19.456825 (+ 2807us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:37:19.456830 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:37:19.456833 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:37:19.456863 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:37:19.458700 (+ 1837us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:37:19.459340 (+ 640us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:37:19.459343 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:37:19.459345 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:37:19.459440 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:37:19.459705 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:37:19.459708 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:37:19.459710 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:37:19.459928 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:37:19.460036 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0809 09:37:19.460343 (+ 307us) server_negotiation.cc:299] Negotiation successful
0809 09:37:19.460476 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260809 09:37:19.460662 21915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:37:19.449780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:35987 (local address 127.0.0.1:41998)
0809 09:37:19.450214 (+ 434us) negotiation.cc:107] Waiting for socket to connect
0809 09:37:19.450256 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 09:37:19.450398 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:37:19.450800 (+ 402us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:37:19.450804 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:37:19.451157 (+ 353us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:37:19.451341 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:37:19.451350 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:37:19.452143 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:37:19.452147 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:37:19.453354 (+ 1207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:37:19.453365 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:37:19.454042 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:37:19.454067 (+ 25us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:37:19.454191 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:37:19.454848 (+ 657us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 09:37:19.454866 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:37:19.456670 (+ 1804us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 09:37:19.458815 (+ 2145us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:37:19.458825 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:37:19.458831 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:37:19.459239 (+ 408us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:37:19.459531 (+ 292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:37:19.459533 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:37:19.459535 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:37:19.459615 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:37:19.460033 (+ 418us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 09:37:19.460038 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 09:37:19.460135 (+ 97us) client_negotiation.cc:769] Sending connection context
0809 09:37:19.460254 (+ 119us) client_negotiation.cc:240] Negotiation successful
0809 09:37:19.460431 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":288,"thread_start_us":176,"threads_started":1}
I20260809 09:37:19.461966 21909 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.125.189" port: 44001 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 09:37:19 dist-test-slave-7kzw krb5kdc[21558](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268239, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.188@KRBTEST.COM for kudu/127.20.125.189@KRBTEST.COM
Aug 09 09:37:19 dist-test-slave-7kzw krb5kdc[21558](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268238, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.125.190@KRBTEST.COM
I20260809 09:37:19.471539 21915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:37:19.462527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.189:44001 (local address 127.0.0.1:49080)
0809 09:37:19.462629 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0809 09:37:19.462645 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 09:37:19.462737 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:37:19.463054 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:37:19.463056 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:37:19.463271 (+ 215us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:37:19.463407 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:37:19.463414 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:37:19.464460 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:37:19.464464 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:37:19.465187 (+ 723us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:37:19.465197 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:37:19.465826 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:37:19.465830 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:37:19.465937 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:37:19.466424 (+ 487us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 09:37:19.466443 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:37:19.467770 (+ 1327us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 09:37:19.469837 (+ 2067us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:37:19.469840 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:37:19.469843 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:37:19.470150 (+ 307us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:37:19.470500 (+ 350us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:37:19.470504 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:37:19.470506 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:37:19.470598 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:37:19.471002 (+ 404us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 09:37:19.471005 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 09:37:19.471107 (+ 102us) client_negotiation.cc:769] Sending connection context
0809 09:37:19.471222 (+ 115us) client_negotiation.cc:240] Negotiation successful
0809 09:37:19.471360 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260809 09:37:19.471565 21800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:37:19.462645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49080 (local address 127.20.125.189:44001)
0809 09:37:19.462799 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0809 09:37:19.462804 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 09:37:19.462820 (+ 16us) server_negotiation.cc:407] Connection header received
0809 09:37:19.462869 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:37:19.462872 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:37:19.462923 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:37:19.463021 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:37:19.463541 (+ 520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:37:19.464353 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:37:19.465302 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:37:19.465668 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:37:19.465808 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:37:19.467924 (+ 2116us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:37:19.467929 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:37:19.467932 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:37:19.467961 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:37:19.469665 (+ 1704us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:37:19.470260 (+ 595us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:37:19.470267 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:37:19.470269 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:37:19.470363 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:37:19.470704 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:37:19.470708 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:37:19.470710 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:37:19.470896 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:37:19.471013 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0809 09:37:19.471294 (+ 281us) server_negotiation.cc:299] Negotiation successful
0809 09:37:19.471415 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260809 09:37:19.472216 21909 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.125.188" port: 43753 } has no permanent_uuid. Determining permanent_uuid...
I20260809 09:37:19.472666 21688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:37:19.462785 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42010 (local address 127.20.125.190:35987)
0809 09:37:19.462924 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0809 09:37:19.462929 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 09:37:19.463113 (+ 184us) server_negotiation.cc:407] Connection header received
0809 09:37:19.463254 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:37:19.463258 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:37:19.463305 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:37:19.463404 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:37:19.464143 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:37:19.464827 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:37:19.465835 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:37:19.466216 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:37:19.466347 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:37:19.469189 (+ 2842us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:37:19.469192 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:37:19.469194 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:37:19.469217 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:37:19.470843 (+ 1626us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:37:19.471290 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:37:19.471295 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:37:19.471297 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:37:19.471384 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:37:19.471741 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:37:19.471746 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:37:19.471749 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:37:19.471974 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:37:19.472225 (+ 251us) server_negotiation.cc:1035] Waiting for connection context
0809 09:37:19.472347 (+ 122us) server_negotiation.cc:299] Negotiation successful
0809 09:37:19.472471 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
Aug 09 09:37:19 dist-test-slave-7kzw krb5kdc[21558](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268239, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.188@KRBTEST.COM for kudu/127.20.125.188@KRBTEST.COM
Aug 09 09:37:19 dist-test-slave-7kzw krb5kdc[21558](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268238, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.125.189@KRBTEST.COM
I20260809 09:37:19.482434 21915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:37:19.472732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.188:43753 (local address 127.0.0.1:50984)
0809 09:37:19.472828 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0809 09:37:19.472844 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 09:37:19.472921 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:37:19.473254 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:37:19.473256 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:37:19.473411 (+ 155us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:37:19.473507 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:37:19.473515 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:37:19.474933 (+ 1418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:37:19.474936 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:37:19.475817 (+ 881us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:37:19.475827 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:37:19.476483 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:37:19.476486 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:37:19.476549 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:37:19.477037 (+ 488us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 09:37:19.477056 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:37:19.478824 (+ 1768us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 09:37:19.480881 (+ 2057us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:37:19.480887 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:37:19.480889 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:37:19.481088 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:37:19.481382 (+ 294us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:37:19.481385 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:37:19.481387 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:37:19.481472 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:37:19.481847 (+ 375us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 09:37:19.481850 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 09:37:19.481935 (+ 85us) client_negotiation.cc:769] Sending connection context
0809 09:37:19.482050 (+ 115us) client_negotiation.cc:240] Negotiation successful
0809 09:37:19.482226 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28}
I20260809 09:37:19.482442 21914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:37:19.472781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50984 (local address 127.20.125.188:43753)
0809 09:37:19.472905 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0809 09:37:19.472912 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 09:37:19.472929 (+ 17us) server_negotiation.cc:407] Connection header received
0809 09:37:19.473022 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:37:19.473025 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:37:19.473069 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:37:19.473151 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:37:19.473630 (+ 479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:37:19.474818 (+ 1188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:37:19.475964 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:37:19.476365 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:37:19.476484 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:37:19.479001 (+ 2517us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:37:19.479004 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:37:19.479007 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:37:19.479033 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:37:19.480774 (+ 1741us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:37:19.481187 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:37:19.481193 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:37:19.481199 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:37:19.481281 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:37:19.481570 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:37:19.481573 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:37:19.481575 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:37:19.481750 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:37:19.481859 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0809 09:37:19.482111 (+ 252us) server_negotiation.cc:299] Negotiation successful
0809 09:37:19.482290 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":57,"server-negotiator.queue_time_us":48}
I20260809 09:37:19.484102 21800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:37:19.474063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49090 (local address 127.20.125.189:44001)
0809 09:37:19.474194 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0809 09:37:19.474199 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 09:37:19.474214 (+ 15us) server_negotiation.cc:407] Connection header received
0809 09:37:19.474295 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:37:19.474298 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:37:19.474341 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:37:19.474423 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:37:19.475061 (+ 638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:37:19.475861 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:37:19.477046 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:37:19.477398 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:37:19.477535 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:37:19.480592 (+ 3057us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:37:19.480595 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:37:19.480598 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:37:19.480620 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:37:19.482164 (+ 1544us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:37:19.482755 (+ 591us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:37:19.482759 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:37:19.482761 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:37:19.482862 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:37:19.483164 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:37:19.483168 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:37:19.483169 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:37:19.483416 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:37:19.483710 (+ 294us) server_negotiation.cc:1035] Waiting for connection context
0809 09:37:19.483807 (+ 97us) server_negotiation.cc:299] Negotiation successful
0809 09:37:19.483933 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260809 09:37:19.486131 21909 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3b07b93bacc34ee69578538bc8afa102: Bootstrap starting.
I20260809 09:37:19.487623 20982 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 09:37:19.489125 21909 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3b07b93bacc34ee69578538bc8afa102: Neither blocks nor log segments found. Creating new log.
I20260809 09:37:19.490079 21909 log.cc:824] T 00000000000000000000000000000000 P 3b07b93bacc34ee69578538bc8afa102: Log is configured to *not* fsync() on all Append() calls
I20260809 09:37:19.488936 21646 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "b673fd5cc908430ca7884785433738ce"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42010
I20260809 09:37:19.492487 21909 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3b07b93bacc34ee69578538bc8afa102: No bootstrap required, opened a new log
I20260809 09:37:19.497022 21909 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3b07b93bacc34ee69578538bc8afa102 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b673fd5cc908430ca7884785433738ce" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 35987 } } peers { permanent_uuid: "151817845efc4227a6d3735174ea5b0e" member_type: VOTER last_known_addr { host: "127.20.125.189" port: 44001 } } peers { permanent_uuid: "3b07b93bacc34ee69578538bc8afa102" member_type: VOTER last_known_addr { host: "127.20.125.188" port: 43753 } }
I20260809 09:37:19.497298 21909 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3b07b93bacc34ee69578538bc8afa102 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b07b93bacc34ee69578538bc8afa102, State: Initialized, Role: FOLLOWER
I20260809 09:37:19.497714 21909 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3b07b93bacc34ee69578538bc8afa102 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b673fd5cc908430ca7884785433738ce" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 35987 } } peers { permanent_uuid: "151817845efc4227a6d3735174ea5b0e" member_type: VOTER last_known_addr { host: "127.20.125.189" port: 44001 } } peers { permanent_uuid: "3b07b93bacc34ee69578538bc8afa102" member_type: VOTER last_known_addr { host: "127.20.125.188" port: 43753 } }
I20260809 09:37:19.498243 21918 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3b07b93bacc34ee69578538bc8afa102 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b673fd5cc908430ca7884785433738ce" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 35987 } } peers { permanent_uuid: "151817845efc4227a6d3735174ea5b0e" member_type: VOTER last_known_addr { host: "127.20.125.189" port: 44001 } } peers { permanent_uuid: "3b07b93bacc34ee69578538bc8afa102" member_type: VOTER last_known_addr { host: "127.20.125.188" port: 43753 } } }
I20260809 09:37:19.498391 21918 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3b07b93bacc34ee69578538bc8afa102 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 09:37:19.499221 21909 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3b07b93bacc34ee69578538bc8afa102 [sys.catalog]: configured and running, proceeding with master startup.
W20260809 09:37:19.504575 21934 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3b07b93bacc34ee69578538bc8afa102: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 09:37:19.504662 21934 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 09:37:19.503474 21806 master_runner.cc:186] Error getting master registration for 127.20.125.190:35987: OK, instance_id { permanent_uuid: "b673fd5cc908430ca7884785433738ce" instance_seqno: 1786268238925194 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 09:37:19.507038 21806 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 09:37:19.564922 21793 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.125.188" port: 43753 } attempt: 4
I20260809 09:37:19.568847 21683 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.125.188" port: 43753 } attempt: 4
Aug 09 09:37:19 dist-test-slave-7kzw krb5kdc[21558](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268239, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.189@KRBTEST.COM for kudu/127.20.125.188@KRBTEST.COM
I20260809 09:37:19.576040 21802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:37:19.565541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.188:43753 (local address 127.0.0.1:50994)
0809 09:37:19.565763 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0809 09:37:19.565778 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 09:37:19.565874 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:37:19.566164 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:37:19.566167 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:37:19.566411 (+ 244us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:37:19.566547 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:37:19.566555 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:37:19.567716 (+ 1161us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:37:19.567720 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:37:19.568580 (+ 860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:37:19.568592 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:37:19.569302 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:37:19.569306 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:37:19.569407 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:37:19.569960 (+ 553us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 09:37:19.569976 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:37:19.571748 (+ 1772us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 09:37:19.573689 (+ 1941us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:37:19.573693 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:37:19.573696 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:37:19.573975 (+ 279us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:37:19.574680 (+ 705us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:37:19.574697 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:37:19.574700 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:37:19.574802 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:37:19.575405 (+ 603us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 09:37:19.575409 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 09:37:19.575544 (+ 135us) client_negotiation.cc:769] Sending connection context
0809 09:37:19.575665 (+ 121us) client_negotiation.cc:240] Negotiation successful
0809 09:37:19.575814 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131}
I20260809 09:37:19.576045 21914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:37:19.565698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50994 (local address 127.20.125.188:43753)
0809 09:37:19.565837 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0809 09:37:19.565841 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 09:37:19.565858 (+ 17us) server_negotiation.cc:407] Connection header received
0809 09:37:19.565977 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:37:19.565980 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:37:19.566026 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:37:19.566111 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:37:19.566690 (+ 579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:37:19.567575 (+ 885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:37:19.568719 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:37:19.569099 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:37:19.569228 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:37:19.571900 (+ 2672us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:37:19.571904 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:37:19.571907 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:37:19.571935 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:37:19.573552 (+ 1617us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:37:19.574099 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:37:19.574103 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:37:19.574105 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:37:19.574199 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:37:19.574927 (+ 728us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:37:19.574931 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:37:19.574933 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:37:19.575243 (+ 310us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:37:19.575350 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0809 09:37:19.575760 (+ 410us) server_negotiation.cc:299] Negotiation successful
0809 09:37:19.575881 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
Aug 09 09:37:19 dist-test-slave-7kzw krb5kdc[21558](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268238, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.190@KRBTEST.COM for kudu/127.20.125.188@KRBTEST.COM
I20260809 09:37:19.580552 21793 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 151817845efc4227a6d3735174ea5b0e: Bootstrap starting.
I20260809 09:37:19.580814 21689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:37:19.569363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.188:43753 (local address 127.0.0.1:51002)
0809 09:37:19.569509 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0809 09:37:19.569526 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 09:37:19.569619 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:37:19.570328 (+ 709us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:37:19.570332 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:37:19.570588 (+ 256us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:37:19.570713 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:37:19.570725 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:37:19.571912 (+ 1187us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:37:19.571916 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:37:19.572879 (+ 963us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:37:19.572894 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:37:19.574335 (+ 1441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:37:19.574339 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:37:19.574438 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:37:19.574957 (+ 519us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 09:37:19.574978 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:37:19.576574 (+ 1596us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 09:37:19.578944 (+ 2370us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:37:19.578947 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:37:19.578949 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:37:19.579237 (+ 288us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:37:19.579544 (+ 307us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:37:19.579547 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:37:19.579550 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:37:19.579683 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:37:19.580212 (+ 529us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 09:37:19.580216 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 09:37:19.580320 (+ 104us) client_negotiation.cc:769] Sending connection context
0809 09:37:19.580430 (+ 110us) client_negotiation.cc:240] Negotiation successful
0809 09:37:19.580566 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57}
I20260809 09:37:19.580839 21937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:37:19.569663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51002 (local address 127.20.125.188:43753)
0809 09:37:19.570084 (+ 421us) server_negotiation.cc:206] Beginning negotiation
0809 09:37:19.570093 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 09:37:19.570107 (+ 14us) server_negotiation.cc:407] Connection header received
0809 09:37:19.570146 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:37:19.570151 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:37:19.570192 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:37:19.570288 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:37:19.570869 (+ 581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:37:19.571766 (+ 897us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:37:19.573674 (+ 1908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:37:19.574098 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:37:19.574210 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:37:19.576755 (+ 2545us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:37:19.576758 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:37:19.576761 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:37:19.576788 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:37:19.578796 (+ 2008us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:37:19.579346 (+ 550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:37:19.579349 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:37:19.579351 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:37:19.579441 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:37:19.579847 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:37:19.579851 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:37:19.579852 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:37:19.580099 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:37:19.580218 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0809 09:37:19.580510 (+ 292us) server_negotiation.cc:299] Negotiation successful
0809 09:37:19.580646 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":343,"thread_start_us":167,"threads_started":1}
I20260809 09:37:19.583595 21793 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 151817845efc4227a6d3735174ea5b0e: Neither blocks nor log segments found. Creating new log.
I20260809 09:37:19.584335 21683 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b673fd5cc908430ca7884785433738ce: Bootstrap starting.
I20260809 09:37:19.584583 21793 log.cc:824] T 00000000000000000000000000000000 P 151817845efc4227a6d3735174ea5b0e: Log is configured to *not* fsync() on all Append() calls
I20260809 09:37:19.586781 21793 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 151817845efc4227a6d3735174ea5b0e: No bootstrap required, opened a new log
I20260809 09:37:19.586933 21683 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b673fd5cc908430ca7884785433738ce: Neither blocks nor log segments found. Creating new log.
I20260809 09:37:19.587792 21683 log.cc:824] T 00000000000000000000000000000000 P b673fd5cc908430ca7884785433738ce: Log is configured to *not* fsync() on all Append() calls
I20260809 09:37:19.590059 21683 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b673fd5cc908430ca7884785433738ce: No bootstrap required, opened a new log
I20260809 09:37:19.591507 21793 raft_consensus.cc:348] T 00000000000000000000000000000000 P 151817845efc4227a6d3735174ea5b0e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b673fd5cc908430ca7884785433738ce" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 35987 } } peers { permanent_uuid: "151817845efc4227a6d3735174ea5b0e" member_type: VOTER last_known_addr { host: "127.20.125.189" port: 44001 } } peers { permanent_uuid: "3b07b93bacc34ee69578538bc8afa102" member_type: VOTER last_known_addr { host: "127.20.125.188" port: 43753 } }
I20260809 09:37:19.591778 21793 raft_consensus.cc:729] T 00000000000000000000000000000000 P 151817845efc4227a6d3735174ea5b0e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 151817845efc4227a6d3735174ea5b0e, State: Initialized, Role: FOLLOWER
I20260809 09:37:19.592214 21793 consensus_queue.cc:260] T 00000000000000000000000000000000 P 151817845efc4227a6d3735174ea5b0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b673fd5cc908430ca7884785433738ce" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 35987 } } peers { permanent_uuid: "151817845efc4227a6d3735174ea5b0e" member_type: VOTER last_known_addr { host: "127.20.125.189" port: 44001 } } peers { permanent_uuid: "3b07b93bacc34ee69578538bc8afa102" member_type: VOTER last_known_addr { host: "127.20.125.188" port: 43753 } }
I20260809 09:37:19.592880 21942 sys_catalog.cc:455] T 00000000000000000000000000000000 P 151817845efc4227a6d3735174ea5b0e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b673fd5cc908430ca7884785433738ce" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 35987 } } peers { permanent_uuid: "151817845efc4227a6d3735174ea5b0e" member_type: VOTER last_known_addr { host: "127.20.125.189" port: 44001 } } peers { permanent_uuid: "3b07b93bacc34ee69578538bc8afa102" member_type: VOTER last_known_addr { host: "127.20.125.188" port: 43753 } } }
I20260809 09:37:19.593309 21942 sys_catalog.cc:458] T 00000000000000000000000000000000 P 151817845efc4227a6d3735174ea5b0e [sys.catalog]: This master's current role is: FOLLOWER
I20260809 09:37:19.593545 21683 raft_consensus.cc:348] T 00000000000000000000000000000000 P b673fd5cc908430ca7884785433738ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b673fd5cc908430ca7884785433738ce" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 35987 } } peers { permanent_uuid: "151817845efc4227a6d3735174ea5b0e" member_type: VOTER last_known_addr { host: "127.20.125.189" port: 44001 } } peers { permanent_uuid: "3b07b93bacc34ee69578538bc8afa102" member_type: VOTER last_known_addr { host: "127.20.125.188" port: 43753 } }
I20260809 09:37:19.593766 21683 raft_consensus.cc:729] T 00000000000000000000000000000000 P b673fd5cc908430ca7884785433738ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b673fd5cc908430ca7884785433738ce, State: Initialized, Role: FOLLOWER
I20260809 09:37:19.593858 21793 sys_catalog.cc:564] T 00000000000000000000000000000000 P 151817845efc4227a6d3735174ea5b0e [sys.catalog]: configured and running, proceeding with master startup.
I20260809 09:37:19.594061 21683 consensus_queue.cc:260] T 00000000000000000000000000000000 P b673fd5cc908430ca7884785433738ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b673fd5cc908430ca7884785433738ce" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 35987 } } peers { permanent_uuid: "151817845efc4227a6d3735174ea5b0e" member_type: VOTER last_known_addr { host: "127.20.125.189" port: 44001 } } peers { permanent_uuid: "3b07b93bacc34ee69578538bc8afa102" member_type: VOTER last_known_addr { host: "127.20.125.188" port: 43753 } }
I20260809 09:37:19.594668 21943 sys_catalog.cc:455] T 00000000000000000000000000000000 P b673fd5cc908430ca7884785433738ce [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b673fd5cc908430ca7884785433738ce" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 35987 } } peers { permanent_uuid: "151817845efc4227a6d3735174ea5b0e" member_type: VOTER last_known_addr { host: "127.20.125.189" port: 44001 } } peers { permanent_uuid: "3b07b93bacc34ee69578538bc8afa102" member_type: VOTER last_known_addr { host: "127.20.125.188" port: 43753 } } }
I20260809 09:37:19.595137 21943 sys_catalog.cc:458] T 00000000000000000000000000000000 P b673fd5cc908430ca7884785433738ce [sys.catalog]: This master's current role is: FOLLOWER
I20260809 09:37:19.595279 21683 sys_catalog.cc:564] T 00000000000000000000000000000000 P b673fd5cc908430ca7884785433738ce [sys.catalog]: configured and running, proceeding with master startup.
I20260809 09:37:19.595343 21646 raft_consensus.cc:482] T 00000000000000000000000000000000 P b673fd5cc908430ca7884785433738ce [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 09:37:19.597375 21646 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b673fd5cc908430ca7884785433738ce [term 0 FOLLOWER]: Advancing to term 1
I20260809 09:37:19.598834 21646 raft_consensus.cc:504] T 00000000000000000000000000000000 P b673fd5cc908430ca7884785433738ce [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b673fd5cc908430ca7884785433738ce" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 35987 } } peers { permanent_uuid: "151817845efc4227a6d3735174ea5b0e" member_type: VOTER last_known_addr { host: "127.20.125.189" port: 44001 } } peers { permanent_uuid: "3b07b93bacc34ee69578538bc8afa102" member_type: VOTER last_known_addr { host: "127.20.125.188" port: 43753 } }
I20260809 09:37:19.599778 21646 leader_election.cc:288] T 00000000000000000000000000000000 P b673fd5cc908430ca7884785433738ce [CANDIDATE]: Term 1 election: Requested vote from peers 151817845efc4227a6d3735174ea5b0e (127.20.125.189:44001), 3b07b93bacc34ee69578538bc8afa102 (127.20.125.188:43753)
I20260809 09:37:19.600550 21870 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b673fd5cc908430ca7884785433738ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "3b07b93bacc34ee69578538bc8afa102"
I20260809 09:37:19.600710 21759 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b673fd5cc908430ca7884785433738ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "151817845efc4227a6d3735174ea5b0e"
I20260809 09:37:19.600785 21870 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3b07b93bacc34ee69578538bc8afa102 [term 0 FOLLOWER]: Advancing to term 1
I20260809 09:37:19.600943 21759 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 151817845efc4227a6d3735174ea5b0e [term 0 FOLLOWER]: Advancing to term 1
I20260809 09:37:19.602105 21870 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 3b07b93bacc34ee69578538bc8afa102 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b673fd5cc908430ca7884785433738ce in term 1.
I20260809 09:37:19.602309 21759 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 151817845efc4227a6d3735174ea5b0e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b673fd5cc908430ca7884785433738ce in term 1.
I20260809 09:37:19.598829 21696 master_runner.cc:186] Error getting master registration for 127.20.125.190:35987: OK, instance_id { permanent_uuid: "b673fd5cc908430ca7884785433738ce" instance_seqno: 1786268238925194 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 09:37:19.602707 21696 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 09:37:19.602820 21609 leader_election.cc:302] T 00000000000000000000000000000000 P b673fd5cc908430ca7884785433738ce [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 151817845efc4227a6d3735174ea5b0e, 3b07b93bacc34ee69578538bc8afa102, b673fd5cc908430ca7884785433738ce; no voters:
I20260809 09:37:19.603135 21943 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b673fd5cc908430ca7884785433738ce [term 1 FOLLOWER]: Leader election won for term 1
I20260809 09:37:19.603394 21943 raft_consensus.cc:686] T 00000000000000000000000000000000 P b673fd5cc908430ca7884785433738ce [term 1 LEADER]: Becoming Leader. State: Replica: b673fd5cc908430ca7884785433738ce, State: Running, Role: LEADER
W20260809 09:37:19.603688 21958 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 151817845efc4227a6d3735174ea5b0e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 09:37:19.603751 21958 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 09:37:19.603741 21943 consensus_queue.cc:237] T 00000000000000000000000000000000 P b673fd5cc908430ca7884785433738ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b673fd5cc908430ca7884785433738ce" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 35987 } } peers { permanent_uuid: "151817845efc4227a6d3735174ea5b0e" member_type: VOTER last_known_addr { host: "127.20.125.189" port: 44001 } } peers { permanent_uuid: "3b07b93bacc34ee69578538bc8afa102" member_type: VOTER last_known_addr { host: "127.20.125.188" port: 43753 } }
I20260809 09:37:19.605675 21943 sys_catalog.cc:455] T 00000000000000000000000000000000 P b673fd5cc908430ca7884785433738ce [sys.catalog]: SysCatalogTable state changed. Reason: New leader b673fd5cc908430ca7884785433738ce. Latest consensus state: current_term: 1 leader_uuid: "b673fd5cc908430ca7884785433738ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b673fd5cc908430ca7884785433738ce" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 35987 } } peers { permanent_uuid: "151817845efc4227a6d3735174ea5b0e" member_type: VOTER last_known_addr { host: "127.20.125.189" port: 44001 } } peers { permanent_uuid: "3b07b93bacc34ee69578538bc8afa102" member_type: VOTER last_known_addr { host: "127.20.125.188" port: 43753 } } }
I20260809 09:37:19.605795 21943 sys_catalog.cc:458] T 00000000000000000000000000000000 P b673fd5cc908430ca7884785433738ce [sys.catalog]: This master's current role is: LEADER
I20260809 09:37:19.606580 21978 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 09:37:19.610074 21978 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 09:37:19.619663 21688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:37:19.607470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42012 (local address 127.20.125.190:35987)
0809 09:37:19.607582 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0809 09:37:19.607587 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 09:37:19.609540 (+ 1953us) server_negotiation.cc:407] Connection header received
0809 09:37:19.609748 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:37:19.609751 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:37:19.609798 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:37:19.609847 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:37:19.611294 (+ 1447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:37:19.612063 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:37:19.613314 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:37:19.613659 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:37:19.613751 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:37:19.615789 (+ 2038us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:37:19.615793 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:37:19.615796 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:37:19.615823 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:37:19.617573 (+ 1750us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:37:19.618084 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:37:19.618088 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:37:19.618089 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:37:19.618191 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:37:19.618532 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:37:19.618535 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:37:19.618537 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:37:19.618777 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:37:19.618921 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0809 09:37:19.619364 (+ 443us) server_negotiation.cc:299] Negotiation successful
0809 09:37:19.619489 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260809 09:37:19.623557 21870 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 3b07b93bacc34ee69578538bc8afa102 [term 1 FOLLOWER]: Refusing update from remote peer b673fd5cc908430ca7884785433738ce: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 09:37:19.624454 21961 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b673fd5cc908430ca7884785433738ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b07b93bacc34ee69578538bc8afa102" member_type: VOTER last_known_addr { host: "127.20.125.188" port: 43753 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 09:37:19.628473 21800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:37:19.608305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49104 (local address 127.20.125.189:44001)
0809 09:37:19.608506 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0809 09:37:19.608511 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 09:37:19.611187 (+ 2676us) server_negotiation.cc:407] Connection header received
0809 09:37:19.611245 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:37:19.611248 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:37:19.611295 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:37:19.611377 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:37:19.618801 (+ 7424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:37:19.619648 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:37:19.622009 (+ 2361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:37:19.622361 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:37:19.622440 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:37:19.624512 (+ 2072us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:37:19.624515 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:37:19.624518 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:37:19.624546 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:37:19.626298 (+ 1752us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:37:19.626886 (+ 588us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:37:19.626889 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:37:19.626891 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:37:19.626982 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:37:19.627340 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:37:19.627346 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:37:19.627351 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:37:19.627616 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:37:19.627743 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0809 09:37:19.628132 (+ 389us) server_negotiation.cc:299] Negotiation successful
0809 09:37:19.628291 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113}
I20260809 09:37:19.629446 21918 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3b07b93bacc34ee69578538bc8afa102 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b673fd5cc908430ca7884785433738ce. Latest consensus state: current_term: 1 leader_uuid: "b673fd5cc908430ca7884785433738ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b673fd5cc908430ca7884785433738ce" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 35987 } } peers { permanent_uuid: "151817845efc4227a6d3735174ea5b0e" member_type: VOTER last_known_addr { host: "127.20.125.189" port: 44001 } } peers { permanent_uuid: "3b07b93bacc34ee69578538bc8afa102" member_type: VOTER last_known_addr { host: "127.20.125.188" port: 43753 } } }
I20260809 09:37:19.629590 21918 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3b07b93bacc34ee69578538bc8afa102 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 09:37:19.630839 21759 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 151817845efc4227a6d3735174ea5b0e [term 1 FOLLOWER]: Refusing update from remote peer b673fd5cc908430ca7884785433738ce: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 09:37:19.631717 21961 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b673fd5cc908430ca7884785433738ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "151817845efc4227a6d3735174ea5b0e" member_type: VOTER last_known_addr { host: "127.20.125.189" port: 44001 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 09:37:19.632331 21937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:37:19.608907 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51010 (local address 127.20.125.188:43753)
0809 09:37:19.609016 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0809 09:37:19.609021 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 09:37:19.614751 (+ 5730us) server_negotiation.cc:407] Connection header received
0809 09:37:19.623388 (+ 8637us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:37:19.623393 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:37:19.623457 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:37:19.623515 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:37:19.624386 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:37:19.625210 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:37:19.626457 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:37:19.626877 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:37:19.626956 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:37:19.628617 (+ 1661us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:37:19.628622 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:37:19.628624 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:37:19.628657 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:37:19.630391 (+ 1734us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:37:19.630932 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:37:19.630936 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:37:19.630938 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:37:19.631025 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:37:19.631326 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:37:19.631329 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:37:19.631331 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:37:19.631540 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:37:19.631662 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0809 09:37:19.632041 (+ 379us) server_negotiation.cc:299] Negotiation successful
0809 09:37:19.632165 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260809 09:37:19.636806 21942 sys_catalog.cc:455] T 00000000000000000000000000000000 P 151817845efc4227a6d3735174ea5b0e [sys.catalog]: SysCatalogTable state changed. Reason: New leader b673fd5cc908430ca7884785433738ce. Latest consensus state: current_term: 1 leader_uuid: "b673fd5cc908430ca7884785433738ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b673fd5cc908430ca7884785433738ce" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 35987 } } peers { permanent_uuid: "151817845efc4227a6d3735174ea5b0e" member_type: VOTER last_known_addr { host: "127.20.125.189" port: 44001 } } peers { permanent_uuid: "3b07b93bacc34ee69578538bc8afa102" member_type: VOTER last_known_addr { host: "127.20.125.188" port: 43753 } } }
I20260809 09:37:19.636945 21942 sys_catalog.cc:458] T 00000000000000000000000000000000 P 151817845efc4227a6d3735174ea5b0e [sys.catalog]: This master's current role is: FOLLOWER
I20260809 09:37:19.639081 21961 sys_catalog.cc:455] T 00000000000000000000000000000000 P b673fd5cc908430ca7884785433738ce [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b673fd5cc908430ca7884785433738ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b673fd5cc908430ca7884785433738ce" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 35987 } } peers { permanent_uuid: "151817845efc4227a6d3735174ea5b0e" member_type: VOTER last_known_addr { host: "127.20.125.189" port: 44001 } } peers { permanent_uuid: "3b07b93bacc34ee69578538bc8afa102" member_type: VOTER last_known_addr { host: "127.20.125.188" port: 43753 } } }
I20260809 09:37:19.639232 21961 sys_catalog.cc:458] T 00000000000000000000000000000000 P b673fd5cc908430ca7884785433738ce [sys.catalog]: This master's current role is: LEADER
I20260809 09:37:19.642827 21918 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3b07b93bacc34ee69578538bc8afa102 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b673fd5cc908430ca7884785433738ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b673fd5cc908430ca7884785433738ce" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 35987 } } peers { permanent_uuid: "151817845efc4227a6d3735174ea5b0e" member_type: VOTER last_known_addr { host: "127.20.125.189" port: 44001 } } peers { permanent_uuid: "3b07b93bacc34ee69578538bc8afa102" member_type: VOTER last_known_addr { host: "127.20.125.188" port: 43753 } } }
I20260809 09:37:19.642958 21918 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3b07b93bacc34ee69578538bc8afa102 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 09:37:19.643708 21990 mvcc.cc:205] Tried to move back new op lower bound from 7316554709490700288 to 7316554709420613632. Current Snapshot: MvccSnapshot[applied={T|T < 7316554709490700288}]
I20260809 09:37:19.644376 21992 mvcc.cc:205] Tried to move back new op lower bound from 7316554709490700288 to 7316554709420613632. Current Snapshot: MvccSnapshot[applied={T|T < 7316554709490700288}]
I20260809 09:37:19.652956 21978 catalog_manager.cc:1347] Generated new cluster ID: 812b624a36314e7bb2ea9e93f538b73d
I20260809 09:37:19.653045 21978 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 09:37:19.653131 21943 sys_catalog.cc:455] T 00000000000000000000000000000000 P b673fd5cc908430ca7884785433738ce [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b673fd5cc908430ca7884785433738ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b673fd5cc908430ca7884785433738ce" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 35987 } } peers { permanent_uuid: "151817845efc4227a6d3735174ea5b0e" member_type: VOTER last_known_addr { host: "127.20.125.189" port: 44001 } } peers { permanent_uuid: "3b07b93bacc34ee69578538bc8afa102" member_type: VOTER last_known_addr { host: "127.20.125.188" port: 43753 } } }
I20260809 09:37:19.653240 21943 sys_catalog.cc:458] T 00000000000000000000000000000000 P b673fd5cc908430ca7884785433738ce [sys.catalog]: This master's current role is: LEADER
I20260809 09:37:19.654477 21942 sys_catalog.cc:455] T 00000000000000000000000000000000 P 151817845efc4227a6d3735174ea5b0e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b673fd5cc908430ca7884785433738ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b673fd5cc908430ca7884785433738ce" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 35987 } } peers { permanent_uuid: "151817845efc4227a6d3735174ea5b0e" member_type: VOTER last_known_addr { host: "127.20.125.189" port: 44001 } } peers { permanent_uuid: "3b07b93bacc34ee69578538bc8afa102" member_type: VOTER last_known_addr { host: "127.20.125.188" port: 43753 } } }
I20260809 09:37:19.654600 21942 sys_catalog.cc:458] T 00000000000000000000000000000000 P 151817845efc4227a6d3735174ea5b0e [sys.catalog]: This master's current role is: FOLLOWER
I20260809 09:37:19.655009 21998 mvcc.cc:205] Tried to move back new op lower bound from 7316554709490700288 to 7316554709420613632. Current Snapshot: MvccSnapshot[applied={T|T < 7316554709490700288}]
I20260809 09:37:19.678887 21978 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 09:37:19.680364 21978 catalog_manager.cc:1504] Loading token signing keys...
I20260809 09:37:19.707814 21978 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b673fd5cc908430ca7884785433738ce: Generated new TSK 0
I20260809 09:37:19.708885 21978 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260809 09:37:19.742681 21937 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 09:37:19.740976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51014 (local address 127.20.125.188:43753)
0809 09:37:19.741101 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0809 09:37:19.741106 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 09:37:19.741643 (+ 537us) server_negotiation.cc:407] Connection header received
0809 09:37:19.741692 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:37:19.741695 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:37:19.741742 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:37:19.741820 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:37:19.742517 (+ 697us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51014: BlockingRecv error: recv got EOF from 127.0.0.1:51014 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
W20260809 09:37:19.742884 21800 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 09:37:19.741068 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49108 (local address 127.20.125.189:44001)
0809 09:37:19.741196 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0809 09:37:19.741202 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 09:37:19.741775 (+ 573us) server_negotiation.cc:407] Connection header received
0809 09:37:19.741961 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:37:19.741965 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:37:19.742017 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:37:19.742087 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:37:19.742750 (+ 663us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49108: BlockingRecv error: recv got EOF from 127.0.0.1:49108 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
W20260809 09:37:19.743942 21688 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 09:37:19.740540 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42024 (local address 127.20.125.190:35987)
0809 09:37:19.740672 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0809 09:37:19.740677 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 09:37:19.741199 (+ 522us) server_negotiation.cc:407] Connection header received
0809 09:37:19.741431 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:37:19.741435 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:37:19.741491 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:37:19.741549 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:37:19.743819 (+ 2270us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42024: BlockingRecv error: recv got EOF from 127.0.0.1:42024 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
W20260809 09:37:19.747299 21800 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 09:37:19.745896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49120 (local address 127.20.125.189:44001)
0809 09:37:19.746001 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0809 09:37:19.746006 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 09:37:19.746254 (+ 248us) server_negotiation.cc:407] Connection header received
0809 09:37:19.746435 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:37:19.746438 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:37:19.746571 (+ 133us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:37:19.746656 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:37:19.747185 (+ 529us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49120: BlockingRecv error: recv got EOF from 127.0.0.1:49120 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
W20260809 09:37:19.751417 21800 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 09:37:19.750521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49134 (local address 127.20.125.189:44001)
0809 09:37:19.750628 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0809 09:37:19.750633 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 09:37:19.750650 (+ 17us) server_negotiation.cc:407] Connection header received
0809 09:37:19.750708 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:37:19.750711 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:37:19.750759 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:37:19.750851 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:37:19.751300 (+ 449us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49134: BlockingRecv error: recv got EOF from 127.0.0.1:49134 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
W20260809 09:37:19.756373 21800 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 09:37:19.755237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49140 (local address 127.20.125.189:44001)
0809 09:37:19.755336 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0809 09:37:19.755341 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 09:37:19.755590 (+ 249us) server_negotiation.cc:407] Connection header received
0809 09:37:19.755699 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:37:19.755702 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:37:19.755754 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:37:19.755845 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:37:19.756281 (+ 436us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49140: BlockingRecv error: recv got EOF from 127.0.0.1:49140 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
W20260809 09:37:19.763324 21800 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 09:37:19.762052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49156 (local address 127.20.125.189:44001)
0809 09:37:19.762155 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0809 09:37:19.762160 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 09:37:19.762418 (+ 258us) server_negotiation.cc:407] Connection header received
0809 09:37:19.762625 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:37:19.762629 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:37:19.762680 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:37:19.762773 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:37:19.763218 (+ 445us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49156: BlockingRecv error: recv got EOF from 127.0.0.1:49156 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
W20260809 09:37:19.774519 21800 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 09:37:19.773216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49158 (local address 127.20.125.189:44001)
0809 09:37:19.773331 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0809 09:37:19.773336 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 09:37:19.773549 (+ 213us) server_negotiation.cc:407] Connection header received
0809 09:37:19.773714 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:37:19.773717 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:37:19.773775 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:37:19.773885 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:37:19.774391 (+ 506us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49158: BlockingRecv error: recv got EOF from 127.0.0.1:49158 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
W20260809 09:37:19.794417 21800 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 09:37:19.792961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49162 (local address 127.20.125.189:44001)
0809 09:37:19.793093 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0809 09:37:19.793099 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 09:37:19.793395 (+ 296us) server_negotiation.cc:407] Connection header received
0809 09:37:19.793544 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:37:19.793547 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:37:19.793608 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:37:19.793697 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:37:19.794291 (+ 594us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49162: BlockingRecv error: recv got EOF from 127.0.0.1:49162 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
W20260809 09:37:19.830406 21800 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 09:37:19.828960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49178 (local address 127.20.125.189:44001)
0809 09:37:19.829151 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0809 09:37:19.829161 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 09:37:19.829302 (+ 141us) server_negotiation.cc:407] Connection header received
0809 09:37:19.829450 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:37:19.829452 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:37:19.829507 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:37:19.829591 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:37:19.830262 (+ 671us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49178: BlockingRecv error: recv got EOF from 127.0.0.1:49178 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
W20260809 09:37:19.898355 21800 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 09:37:19.896996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49190 (local address 127.20.125.189:44001)
0809 09:37:19.897162 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0809 09:37:19.897167 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 09:37:19.897323 (+ 156us) server_negotiation.cc:407] Connection header received
0809 09:37:19.897497 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:37:19.897500 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:37:19.897558 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:37:19.897658 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:37:19.898180 (+ 522us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49190: BlockingRecv error: recv got EOF from 127.0.0.1:49190 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
W20260809 09:37:20.030208 21800 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 09:37:20.028907 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49192 (local address 127.20.125.189:44001)
0809 09:37:20.029102 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0809 09:37:20.029107 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 09:37:20.029232 (+ 125us) server_negotiation.cc:407] Connection header received
0809 09:37:20.029438 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:37:20.029441 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:37:20.029495 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:37:20.029585 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:37:20.030089 (+ 504us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49192: BlockingRecv error: recv got EOF from 127.0.0.1:49192 (error 108)
Metrics: {"server-negotiator.queue_time_us":89}
W20260809 09:37:20.290386 21800 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 09:37:20.289146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49196 (local address 127.20.125.189:44001)
0809 09:37:20.289308 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0809 09:37:20.289313 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 09:37:20.289334 (+ 21us) server_negotiation.cc:407] Connection header received
0809 09:37:20.289475 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:37:20.289478 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:37:20.289531 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:37:20.289614 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:37:20.290227 (+ 613us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49196: BlockingRecv error: recv got EOF from 127.0.0.1:49196 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
I20260809 09:37:20.506283 21934 catalog_manager.cc:1259] Loaded cluster ID: 812b624a36314e7bb2ea9e93f538b73d
I20260809 09:37:20.506393 21934 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 3b07b93bacc34ee69578538bc8afa102: loading cluster ID for follower catalog manager: success
I20260809 09:37:20.508271 21934 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 3b07b93bacc34ee69578538bc8afa102: acquiring CA information for follower catalog manager: success
I20260809 09:37:20.509552 21934 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 3b07b93bacc34ee69578538bc8afa102: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 09:37:20.605428 21958 catalog_manager.cc:1259] Loaded cluster ID: 812b624a36314e7bb2ea9e93f538b73d
I20260809 09:37:20.605504 21958 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 151817845efc4227a6d3735174ea5b0e: loading cluster ID for follower catalog manager: success
I20260809 09:37:20.607365 21958 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 151817845efc4227a6d3735174ea5b0e: acquiring CA information for follower catalog manager: success
I20260809 09:37:20.608554 21958 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 151817845efc4227a6d3735174ea5b0e: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 09:37:20.809015 22105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:37:20.804512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49210 (local address 127.20.125.189:44001)
0809 09:37:20.804806 (+ 294us) server_negotiation.cc:206] Beginning negotiation
0809 09:37:20.804810 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 09:37:20.804829 (+ 19us) server_negotiation.cc:407] Connection header received
0809 09:37:20.805057 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:37:20.805061 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:37:20.805203 (+ 142us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:37:20.805336 (+ 133us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 09:37:20.805642 (+ 306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:37:20.806335 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:37:20.807332 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:37:20.807586 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:37:20.807706 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:37:20.808075 (+ 369us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 09:37:20.808546 (+ 471us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 09:37:20.808718 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0809 09:37:20.808765 (+ 47us) server_negotiation.cc:299] Negotiation successful
0809 09:37:20.808831 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":216,"thread_start_us":109,"threads_started":1}
I20260809 09:37:20.816567 22112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:37:20.812451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51016 (local address 127.20.125.188:43753)
0809 09:37:20.812717 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0809 09:37:20.812721 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 09:37:20.812850 (+ 129us) server_negotiation.cc:407] Connection header received
0809 09:37:20.813084 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:37:20.813087 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:37:20.813209 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:37:20.813342 (+ 133us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 09:37:20.813559 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:37:20.814084 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:37:20.815002 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:37:20.815246 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:37:20.815357 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:37:20.815671 (+ 314us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 09:37:20.816132 (+ 461us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 09:37:20.816279 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0809 09:37:20.816325 (+ 46us) server_negotiation.cc:299] Negotiation successful
0809 09:37:20.816386 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":194,"thread_start_us":91,"threads_started":1}
I20260809 09:37:20.824707 22105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:37:20.819890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49224 (local address 127.20.125.189:44001)
0809 09:37:20.820089 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0809 09:37:20.820093 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 09:37:20.820420 (+ 327us) server_negotiation.cc:407] Connection header received
0809 09:37:20.820586 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:37:20.820590 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:37:20.820648 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:37:20.820742 (+ 94us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 09:37:20.820960 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:37:20.821831 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:37:20.823217 (+ 1386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:37:20.823598 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:37:20.823735 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:37:20.824070 (+ 335us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 09:37:20.824182 (+ 112us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 09:37:20.824267 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0809 09:37:20.824460 (+ 193us) server_negotiation.cc:299] Negotiation successful
0809 09:37:20.824523 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104}
I20260809 09:37:20.825140 22112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:37:20.819951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51030 (local address 127.20.125.188:43753)
0809 09:37:20.820204 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0809 09:37:20.820210 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 09:37:20.820973 (+ 763us) server_negotiation.cc:407] Connection header received
0809 09:37:20.821211 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:37:20.821216 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:37:20.821274 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:37:20.821342 (+ 68us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 09:37:20.821841 (+ 499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:37:20.822523 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:37:20.823698 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:37:20.824063 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:37:20.824205 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:37:20.824513 (+ 308us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 09:37:20.824612 (+ 99us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 09:37:20.824669 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0809 09:37:20.824902 (+ 233us) server_negotiation.cc:299] Negotiation successful
0809 09:37:20.824966 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":161}
I20260809 09:37:20.825619 22124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:37:20.820040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42036 (local address 127.20.125.190:35987)
0809 09:37:20.820338 (+ 298us) server_negotiation.cc:206] Beginning negotiation
0809 09:37:20.820342 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 09:37:20.820575 (+ 233us) server_negotiation.cc:407] Connection header received
0809 09:37:20.820792 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:37:20.820796 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:37:20.820987 (+ 191us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:37:20.821125 (+ 138us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 09:37:20.821362 (+ 237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:37:20.822388 (+ 1026us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:37:20.823767 (+ 1379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:37:20.824148 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:37:20.824251 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:37:20.824749 (+ 498us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 09:37:20.825130 (+ 381us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 09:37:20.825205 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0809 09:37:20.825375 (+ 170us) server_negotiation.cc:299] Negotiation successful
0809 09:37:20.825435 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":207,"thread_start_us":103,"threads_started":1}
I20260809 09:37:20.829346 20982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu with pid 21576
I20260809 09:37:20.843365 20982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu with pid 21696
I20260809 09:37:20.857795 20982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu with pid 21806
2026-08-09T09:37:20Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (3559 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786267941944493-20982-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 09:37:20 dist-test-slave-7kzw krb5kdc[22129](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 09:37:20 dist-test-slave-7kzw krb5kdc[22129](info): set up 2 sockets
Aug 09 09:37:20 dist-test-slave-7kzw krb5kdc[22129](info): commencing operation
krb5kdc: starting...
W20260809 09:37:22.165534 20982 mini_kdc.cc:121] Time spent starting KDC: real 1.280s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786267941944493-20982-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786267941944493-20982-0/krb5kdc/test-user.keytab.
Aug 09 09:37:22 dist-test-slave-7kzw krb5kdc[22129](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268242, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T09:37:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T09:37:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.125.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.125.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.125.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.190.keytab.
Entry for principal kudu/127.20.125.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.190.keytab.
I20260809 09:37:22.300894 20982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
/tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786267941944493-20982-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786267941944493-20982-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786267941944493-20982-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786267941944493-20982-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.125.190:43161
--webserver_interface=127.20.125.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.190.keytab
--principal=kudu/127.20.125.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.125.148:42931
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.125.190:43161,127.20.125.189:45207,127.20.125.188:46547
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786267941944493-20982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786267941944493-20982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786267941944493-20982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 09:37:22.412853 22149 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 09:37:22.413188 22149 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 09:37:22.413249 22149 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 09:37:22.413322 22149 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 09:37:22.417372 22149 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 09:37:22.417455 22149 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 09:37:22.417491 22149 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 09:37:22.417537 22149 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 09:37:22.417573 22149 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 09:37:22.422878 22149 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.125.148:42931
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786267941944493-20982-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786267941944493-20982-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.125.190:43161,127.20.125.189:45207,127.20.125.188:46547
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.190.keytab
--principal=kudu/127.20.125.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.125.190:43161
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786267941944493-20982-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.125.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22149
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786267941944493-20982-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 08:38:55 UTC on ccycloud.quasar-njjleu.root.comops.site
build id 4575935
I20260809 09:37:22.423978 22149 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 09:37:22.425045 22149 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 09:37:22.433741 22163 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:37:22.433776 22157 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:37:22.433777 22160 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:37:22.434113 22149 server_base.cc:1034] running on GCE node
I20260809 09:37:22.434791 22149 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 09:37:22.435837 22149 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 09:37:22.437104 22149 hybrid_clock.cc:648] HybridClock initialized: now 1786268242437068 us; error 58 us; skew 500 ppm
Aug 09 09:37:22 dist-test-slave-7kzw krb5kdc[22129](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268242, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 09:37:22.444809 22149 init.cc:377] Logged in from keytab as kudu/127.20.125.190@KRBTEST.COM (short username kudu)
I20260809 09:37:22.445945 22149 webserver.cc:460] Webserver started at http://127.20.125.190:33623/ using document root <none> and password file <none>
I20260809 09:37:22.446656 22149 fs_manager.cc:359] Metadata directory not provided
I20260809 09:37:22.446767 22149 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 09:37:22.447029 22149 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 09:37:22.449438 22149 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786267941944493-20982-0/minicluster-data/master-0/data/instance:
uuid: "d78be03a73df4aba99e1df24a60ad797"
format_stamp: "Formatted at 2026-08-09 09:37:22 on dist-test-slave-7kzw"
server_key: "3ff8e2805bfe9ac2b58e9eb4698d278f"
server_key_iv: "4610e6f12486fdb1b6899f6f42505858"
server_key_version: "encryptionkey@0"
I20260809 09:37:22.450094 22149 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786267941944493-20982-0/minicluster-data/master-0/wal/instance:
uuid: "d78be03a73df4aba99e1df24a60ad797"
format_stamp: "Formatted at 2026-08-09 09:37:22 on dist-test-slave-7kzw"
server_key: "3ff8e2805bfe9ac2b58e9eb4698d278f"
server_key_iv: "4610e6f12486fdb1b6899f6f42505858"
server_key_version: "encryptionkey@0"
I20260809 09:37:22.454310 22149 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260809 09:37:22.457123 22179 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 09:37:22.458559 22149 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 09:37:22.458701 22149 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786267941944493-20982-0/minicluster-data/master-0/data,/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786267941944493-20982-0/minicluster-data/master-0/wal
uuid: "d78be03a73df4aba99e1df24a60ad797"
format_stamp: "Formatted at 2026-08-09 09:37:22 on dist-test-slave-7kzw"
server_key: "3ff8e2805bfe9ac2b58e9eb4698d278f"
server_key_iv: "4610e6f12486fdb1b6899f6f42505858"
server_key_version: "encryptionkey@0"
I20260809 09:37:22.458820 22149 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786267941944493-20982-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786267941944493-20982-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786267941944493-20982-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 09:37:22.481866 22149 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 09:37:22.485114 22149 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 09:37:22.485334 22149 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 09:37:22.494776 22149 rpc_server.cc:307] RPC server started. Bound to: 127.20.125.190:43161
I20260809 09:37:22.495101 22256 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.125.190:43161 every 8 connection(s)
I20260809 09:37:22.495844 22149 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786267941944493-20982-0/minicluster-data/master-0/data/info.pb
I20260809 09:37:22.497684 20982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu as pid 22149
I20260809 09:37:22.497835 20982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786267941944493-20982-0/minicluster-data/master-0/wal/instance
I20260809 09:37:22.498169 20982 external_mini_cluster.cc:1428] Setting key 15d2c8aa71d4b0e89fa4b49e43a70da5
I20260809 09:37:22.498894 22257 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 09:37:22.503430 22257 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.125.190" port: 43161 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.20.125.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.125.189@KRBTEST.COM" created.
Aug 09 09:37:22 dist-test-slave-7kzw krb5kdc[22129](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268242, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.190@KRBTEST.COM for kudu/127.20.125.190@KRBTEST.COM
I20260809 09:37:22.519290 22262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:37:22.506976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48296 (local address 127.20.125.190:43161)
0809 09:37:22.507328 (+ 352us) server_negotiation.cc:206] Beginning negotiation
0809 09:37:22.507339 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 09:37:22.507383 (+ 44us) server_negotiation.cc:407] Connection header received
0809 09:37:22.507758 (+ 375us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:37:22.507763 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:37:22.507941 (+ 178us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:37:22.508120 (+ 179us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:37:22.508941 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:37:22.509966 (+ 1025us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:37:22.511084 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:37:22.511502 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:37:22.511668 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:37:22.514349 (+ 2681us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:37:22.514356 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:37:22.514370 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:37:22.514398 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:37:22.516107 (+ 1709us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:37:22.516763 (+ 656us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:37:22.516768 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:37:22.516769 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:37:22.516871 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:37:22.517268 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:37:22.517270 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:37:22.517271 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:37:22.517613 (+ 342us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:37:22.517714 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0809 09:37:22.518264 (+ 550us) server_negotiation.cc:299] Negotiation successful
0809 09:37:22.518455 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":225,"thread_start_us":120,"threads_started":1}
I20260809 09:37:22.519297 22261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:37:22.505913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:43161 (local address 127.0.0.1:48296)
0809 09:37:22.506451 (+ 538us) negotiation.cc:107] Waiting for socket to connect
0809 09:37:22.506548 (+ 97us) client_negotiation.cc:174] Beginning negotiation
0809 09:37:22.507458 (+ 910us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:37:22.508120 (+ 662us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:37:22.508125 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:37:22.508569 (+ 444us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:37:22.508801 (+ 232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:37:22.508813 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:37:22.510086 (+ 1273us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:37:22.510090 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:37:22.510940 (+ 850us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:37:22.510949 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:37:22.511652 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:37:22.511655 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:37:22.511780 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:37:22.512410 (+ 630us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 09:37:22.512427 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:37:22.514179 (+ 1752us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 09:37:22.516235 (+ 2056us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:37:22.516249 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:37:22.516263 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:37:22.516638 (+ 375us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:37:22.516971 (+ 333us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:37:22.516976 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:37:22.516981 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:37:22.517168 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:37:22.517743 (+ 575us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 09:37:22.517753 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 09:37:22.517962 (+ 209us) client_negotiation.cc:769] Sending connection context
0809 09:37:22.518208 (+ 246us) client_negotiation.cc:240] Negotiation successful
0809 09:37:22.518605 (+ 397us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":341,"mutex_wait_us":5,"thread_start_us":166,"threads_started":1}
I20260809 09:37:22.521950 22257 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.125.189" port: 45207 } has no permanent_uuid. Determining permanent_uuid...
I20260809 09:37:22.522713 22261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:37:22.522411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.189:45207 (local address 127.0.0.1:43644)
0809 09:37:22.522553 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0809 09:37:22.522641 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.189:45207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
W20260809 09:37:22.522928 22181 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.20.125.189:45207: connect: Connection refused (error 111)
I20260809 09:37:22.523833 22261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:37:22.523581 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.189:45207 (local address 127.0.0.1:43658)
0809 09:37:22.523693 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0809 09:37:22.523748 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.189:45207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260809 09:37:22.523996 22257 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.125.189:45207: Network error: Client connection negotiation failed: client connection to 127.20.125.189:45207: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.125.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.189.keytab.
Entry for principal kudu/127.20.125.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.189.keytab.
I20260809 09:37:22.530218 20982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
/tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786267941944493-20982-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786267941944493-20982-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786267941944493-20982-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786267941944493-20982-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.125.189:45207
--webserver_interface=127.20.125.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.189.keytab
--principal=kudu/127.20.125.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.125.148:42931
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.125.190:43161,127.20.125.189:45207,127.20.125.188:46547
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786267941944493-20982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786267941944493-20982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786267941944493-20982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 09:37:22.573298 22257 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.125.189" port: 45207 } attempt: 1
I20260809 09:37:22.574347 22261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:37:22.573916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.189:45207 (local address 127.0.0.1:43664)
0809 09:37:22.574125 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0809 09:37:22.574223 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.189:45207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260809 09:37:22.575286 22261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:37:22.574973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.189:45207 (local address 127.0.0.1:43666)
0809 09:37:22.575126 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0809 09:37:22.575198 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.189:45207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260809 09:37:22.575449 22257 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.125.189:45207: Network error: Client connection negotiation failed: client connection to 127.20.125.189:45207: connect: Connection refused (error 111)
I20260809 09:37:22.643658 22257 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.125.189" port: 45207 } attempt: 2
I20260809 09:37:22.644598 22261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:37:22.644249 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.189:45207 (local address 127.0.0.1:43670)
0809 09:37:22.644427 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0809 09:37:22.644503 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.189:45207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260809 09:37:22.645340 22261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:37:22.645124 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.189:45207 (local address 127.0.0.1:43676)
0809 09:37:22.645224 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0809 09:37:22.645275 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.189:45207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260809 09:37:22.645468 22257 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.125.189:45207: Network error: Client connection negotiation failed: client connection to 127.20.125.189:45207: connect: Connection refused (error 111)
W20260809 09:37:22.646276 22268 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 09:37:22.646597 22268 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 09:37:22.646631 22268 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 09:37:22.646668 22268 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 09:37:22.650686 22268 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 09:37:22.650734 22268 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 09:37:22.650753 22268 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 09:37:22.650774 22268 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 09:37:22.650789 22268 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 09:37:22.656061 22268 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.125.148:42931
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786267941944493-20982-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786267941944493-20982-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.20.125.190:43161,127.20.125.189:45207,127.20.125.188:46547
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.189.keytab
--principal=kudu/127.20.125.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.125.189:45207
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786267941944493-20982-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.20.125.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22268
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786267941944493-20982-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 08:38:55 UTC on ccycloud.quasar-njjleu.root.comops.site
build id 4575935
I20260809 09:37:22.657101 22268 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 09:37:22.658115 22268 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 09:37:22.666894 22274 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:37:22.666923 22278 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:37:22.666918 22281 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:37:22.667246 22268 server_base.cc:1034] running on GCE node
I20260809 09:37:22.667865 22268 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 09:37:22.668968 22268 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 09:37:22.670132 22268 hybrid_clock.cc:648] HybridClock initialized: now 1786268242670117 us; error 27 us; skew 500 ppm
Aug 09 09:37:22 dist-test-slave-7kzw krb5kdc[22129](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268242, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 09:37:22.673090 22268 init.cc:377] Logged in from keytab as kudu/127.20.125.189@KRBTEST.COM (short username kudu)
I20260809 09:37:22.674014 22268 webserver.cc:460] Webserver started at http://127.20.125.189:39431/ using document root <none> and password file <none>
I20260809 09:37:22.674654 22268 fs_manager.cc:359] Metadata directory not provided
I20260809 09:37:22.674723 22268 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 09:37:22.674934 22268 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 09:37:22.677003 22268 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786267941944493-20982-0/minicluster-data/master-1/data/instance:
uuid: "222dee7fe32c4ccaa54c34d7a072c9da"
format_stamp: "Formatted at 2026-08-09 09:37:22 on dist-test-slave-7kzw"
server_key: "055c73e84d11aa4963f1f9c71de920d6"
server_key_iv: "82791d8f05ae426f72a3c1202d3f3611"
server_key_version: "encryptionkey@0"
I20260809 09:37:22.677587 22268 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786267941944493-20982-0/minicluster-data/master-1/wal/instance:
uuid: "222dee7fe32c4ccaa54c34d7a072c9da"
format_stamp: "Formatted at 2026-08-09 09:37:22 on dist-test-slave-7kzw"
server_key: "055c73e84d11aa4963f1f9c71de920d6"
server_key_iv: "82791d8f05ae426f72a3c1202d3f3611"
server_key_version: "encryptionkey@0"
I20260809 09:37:22.681355 22268 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.005s
I20260809 09:37:22.683923 22291 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 09:37:22.685115 22268 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 09:37:22.685248 22268 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786267941944493-20982-0/minicluster-data/master-1/data,/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786267941944493-20982-0/minicluster-data/master-1/wal
uuid: "222dee7fe32c4ccaa54c34d7a072c9da"
format_stamp: "Formatted at 2026-08-09 09:37:22 on dist-test-slave-7kzw"
server_key: "055c73e84d11aa4963f1f9c71de920d6"
server_key_iv: "82791d8f05ae426f72a3c1202d3f3611"
server_key_version: "encryptionkey@0"
I20260809 09:37:22.685381 22268 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786267941944493-20982-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786267941944493-20982-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786267941944493-20982-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 09:37:22.701468 22268 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 09:37:22.704912 22268 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 09:37:22.705080 22268 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 09:37:22.712600 22268 rpc_server.cc:307] RPC server started. Bound to: 127.20.125.189:45207
I20260809 09:37:22.712989 22381 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.125.189:45207 every 8 connection(s)
I20260809 09:37:22.713616 22268 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786267941944493-20982-0/minicluster-data/master-1/data/info.pb
I20260809 09:37:22.716408 22382 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 09:37:22.717080 20982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu as pid 22268
I20260809 09:37:22.717263 20982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786267941944493-20982-0/minicluster-data/master-1/wal/instance
I20260809 09:37:22.717615 20982 external_mini_cluster.cc:1428] Setting key 2f7659c2673b806349dbd3ed37c30afc
I20260809 09:37:22.720329 22382 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.125.190" port: 43161 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.20.125.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.125.188@KRBTEST.COM" created.
Aug 09 09:37:22 dist-test-slave-7kzw krb5kdc[22129](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268242, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.189@KRBTEST.COM for kudu/127.20.125.190@KRBTEST.COM
I20260809 09:37:22.735280 22262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:37:22.722266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48302 (local address 127.20.125.190:43161)
0809 09:37:22.722401 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0809 09:37:22.722405 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 09:37:22.723532 (+ 1127us) server_negotiation.cc:407] Connection header received
0809 09:37:22.724466 (+ 934us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:37:22.724469 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:37:22.724529 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:37:22.724629 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:37:22.725745 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:37:22.726272 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:37:22.727510 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:37:22.727763 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:37:22.727879 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:37:22.731001 (+ 3122us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:37:22.731004 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:37:22.731006 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:37:22.731030 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:37:22.732849 (+ 1819us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:37:22.733464 (+ 615us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:37:22.733469 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:37:22.733477 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:37:22.733602 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:37:22.733999 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:37:22.734003 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:37:22.734005 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:37:22.734208 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:37:22.734318 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0809 09:37:22.734940 (+ 622us) server_negotiation.cc:299] Negotiation successful
0809 09:37:22.735083 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260809 09:37:22.736116 22388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:37:22.722837 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:43161 (local address 127.0.0.1:48302)
0809 09:37:22.723384 (+ 547us) negotiation.cc:107] Waiting for socket to connect
0809 09:37:22.723425 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 09:37:22.724267 (+ 842us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:37:22.724797 (+ 530us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:37:22.724805 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:37:22.725153 (+ 348us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:37:22.725600 (+ 447us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:37:22.725611 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:37:22.726398 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:37:22.726402 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:37:22.727386 (+ 984us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:37:22.727396 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:37:22.727916 (+ 520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:37:22.727921 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:37:22.728083 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:37:22.728782 (+ 699us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 09:37:22.728799 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:37:22.730825 (+ 2026us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 09:37:22.732994 (+ 2169us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:37:22.732999 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:37:22.733012 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:37:22.733329 (+ 317us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:37:22.733710 (+ 381us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:37:22.733716 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:37:22.733718 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:37:22.733886 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:37:22.734333 (+ 447us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 09:37:22.734342 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 09:37:22.734667 (+ 325us) client_negotiation.cc:769] Sending connection context
0809 09:37:22.734863 (+ 196us) client_negotiation.cc:240] Negotiation successful
0809 09:37:22.735224 (+ 361us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":350,"thread_start_us":165,"threads_started":1}
I20260809 09:37:22.736613 22257 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.125.189" port: 45207 } attempt: 3
I20260809 09:37:22.737792 22382 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.125.189" port: 45207 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 09:37:22 dist-test-slave-7kzw krb5kdc[22129](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268242, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.190@KRBTEST.COM for kudu/127.20.125.189@KRBTEST.COM
Aug 09 09:37:22 dist-test-slave-7kzw krb5kdc[22129](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268242, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.189@KRBTEST.COM for kudu/127.20.125.189@KRBTEST.COM
I20260809 09:37:22.748277 22261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:37:22.737067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.189:45207 (local address 127.0.0.1:43690)
0809 09:37:22.737680 (+ 613us) negotiation.cc:107] Waiting for socket to connect
0809 09:37:22.737702 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 09:37:22.737807 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:37:22.738370 (+ 563us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:37:22.738374 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:37:22.738595 (+ 221us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:37:22.738766 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:37:22.738776 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:37:22.740103 (+ 1327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:37:22.740105 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:37:22.741661 (+ 1556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:37:22.741670 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:37:22.742340 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:37:22.742343 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:37:22.742457 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:37:22.742968 (+ 511us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 09:37:22.742981 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:37:22.744254 (+ 1273us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 09:37:22.746390 (+ 2136us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:37:22.746395 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:37:22.746397 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:37:22.746743 (+ 346us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:37:22.747132 (+ 389us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:37:22.747137 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:37:22.747139 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:37:22.747236 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:37:22.747799 (+ 563us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 09:37:22.747802 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 09:37:22.747897 (+ 95us) client_negotiation.cc:769] Sending connection context
0809 09:37:22.747988 (+ 91us) client_negotiation.cc:240] Negotiation successful
0809 09:37:22.748147 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":528}
I20260809 09:37:22.748744 22391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:37:22.737273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43690 (local address 127.20.125.189:45207)
0809 09:37:22.737632 (+ 359us) server_negotiation.cc:206] Beginning negotiation
0809 09:37:22.737645 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 09:37:22.737777 (+ 132us) server_negotiation.cc:407] Connection header received
0809 09:37:22.737931 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:37:22.737939 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:37:22.738169 (+ 230us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:37:22.738629 (+ 460us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:37:22.738902 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:37:22.739982 (+ 1080us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:37:22.741780 (+ 1798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:37:22.742178 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:37:22.742335 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:37:22.744383 (+ 2048us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:37:22.744393 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:37:22.744410 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:37:22.744432 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:37:22.746273 (+ 1841us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:37:22.746866 (+ 593us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:37:22.746872 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:37:22.746874 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:37:22.746970 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:37:22.747358 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:37:22.747362 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:37:22.747364 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:37:22.747662 (+ 298us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:37:22.747824 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0809 09:37:22.748228 (+ 404us) server_negotiation.cc:299] Negotiation successful
0809 09:37:22.748432 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":35,"server-negotiator.queue_time_us":250,"thread_start_us":168,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.125.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.188.keytab.
Entry for principal kudu/127.20.125.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.188.keytab.
I20260809 09:37:22.750953 22257 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.125.188" port: 46547 } has no permanent_uuid. Determining permanent_uuid...
I20260809 09:37:22.751291 20982 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
/tmp/dist-test-taskYHpNFG/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786267941944493-20982-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786267941944493-20982-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786267941944493-20982-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786267941944493-20982-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.125.188:46547
--webserver_interface=127.20.125.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.188.keytab
--principal=kudu/127.20.125.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.125.148:42931
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.125.190:43161,127.20.125.189:45207,127.20.125.188:46547
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786267941944493-20982-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786267941944493-20982-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786267941944493-20982-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 09:37:22.751766 22388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:37:22.738308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.189:45207 (local address 127.0.0.1:43700)
0809 09:37:22.738406 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0809 09:37:22.738422 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 09:37:22.738514 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:37:22.739241 (+ 727us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:37:22.739247 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:37:22.739428 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:37:22.739571 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:37:22.739581 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:37:22.740552 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:37:22.740558 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:37:22.741427 (+ 869us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:37:22.741438 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:37:22.742459 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:37:22.742484 (+ 25us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:37:22.742587 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:37:22.745675 (+ 3088us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 09:37:22.745696 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:37:22.747045 (+ 1349us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 09:37:22.749989 (+ 2944us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:37:22.749992 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:37:22.749994 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:37:22.750252 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:37:22.750613 (+ 361us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:37:22.750617 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:37:22.750619 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:37:22.750740 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:37:22.751261 (+ 521us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 09:37:22.751265 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 09:37:22.751368 (+ 103us) client_negotiation.cc:769] Sending connection context
0809 09:37:22.751495 (+ 127us) client_negotiation.cc:240] Negotiation successful
0809 09:37:22.751644 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260809 09:37:22.751938 22261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:37:22.751560 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.188:46547 (local address 127.0.0.1:35574)
0809 09:37:22.751775 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0809 09:37:22.751841 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.188:46547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129}
I20260809 09:37:22.752408 22392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:37:22.738590 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43700 (local address 127.20.125.189:45207)
0809 09:37:22.738905 (+ 315us) server_negotiation.cc:206] Beginning negotiation
0809 09:37:22.738914 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 09:37:22.738926 (+ 12us) server_negotiation.cc:407] Connection header received
0809 09:37:22.738960 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:37:22.738963 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:37:22.739003 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:37:22.739227 (+ 224us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:37:22.739701 (+ 474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:37:22.740436 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:37:22.741631 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:37:22.742180 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:37:22.742459 (+ 279us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:37:22.747210 (+ 4751us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:37:22.747215 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:37:22.747223 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:37:22.747244 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:37:22.749867 (+ 2623us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:37:22.750368 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:37:22.750370 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:37:22.750372 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:37:22.750455 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:37:22.750867 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:37:22.750872 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:37:22.750876 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:37:22.751130 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:37:22.751243 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0809 09:37:22.752087 (+ 844us) server_negotiation.cc:299] Negotiation successful
0809 09:37:22.752215 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":22,"server-negotiator.queue_time_us":238,"thread_start_us":111,"threads_started":1}
I20260809 09:37:22.753268 22261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:37:22.752459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.188:46547 (local address 127.0.0.1:35580)
0809 09:37:22.752958 (+ 499us) negotiation.cc:107] Waiting for socket to connect
0809 09:37:22.753186 (+ 228us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.188:46547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":408}
I20260809 09:37:22.753274 22382 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.125.188" port: 46547 } has no permanent_uuid. Determining permanent_uuid...
W20260809 09:37:22.753429 22257 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.125.188:46547: Network error: Client connection negotiation failed: client connection to 127.20.125.188:46547: connect: Connection refused (error 111)
I20260809 09:37:22.753976 22388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:37:22.753726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.188:46547 (local address 127.0.0.1:35596)
0809 09:37:22.753843 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0809 09:37:22.753922 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.188:46547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260809 09:37:22.754253 22294 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.20.125.188:46547: connect: Connection refused (error 111)
I20260809 09:37:22.755280 22388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:37:22.755060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.188:46547 (local address 127.0.0.1:35612)
0809 09:37:22.755175 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0809 09:37:22.755226 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.188:46547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260809 09:37:22.755505 22382 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.125.188:46547: Network error: Client connection negotiation failed: client connection to 127.20.125.188:46547: connect: Connection refused (error 111)
I20260809 09:37:22.784605 22257 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.125.188" port: 46547 } attempt: 1
I20260809 09:37:22.785537 22261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:37:22.785177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.188:46547 (local address 127.0.0.1:35616)
0809 09:37:22.785362 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0809 09:37:22.785437 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.188:46547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260809 09:37:22.786372 22261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:37:22.786128 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.188:46547 (local address 127.0.0.1:35624)
0809 09:37:22.786248 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0809 09:37:22.786304 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.188:46547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260809 09:37:22.786588 22257 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.125.188:46547: Network error: Client connection negotiation failed: client connection to 127.20.125.188:46547: connect: Connection refused (error 111)
I20260809 09:37:22.804836 22382 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.125.188" port: 46547 } attempt: 1
I20260809 09:37:22.805851 22388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:37:22.805501 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.188:46547 (local address 127.0.0.1:35628)
0809 09:37:22.805688 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0809 09:37:22.805758 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.188:46547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260809 09:37:22.806758 22388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:37:22.806536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.188:46547 (local address 127.0.0.1:35632)
0809 09:37:22.806648 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0809 09:37:22.806696 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.188:46547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260809 09:37:22.806910 22382 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.125.188:46547: Network error: Client connection negotiation failed: client connection to 127.20.125.188:46547: connect: Connection refused (error 111)
I20260809 09:37:22.861815 22257 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.125.188" port: 46547 } attempt: 2
I20260809 09:37:22.862846 22261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:37:22.862433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.188:46547 (local address 127.0.0.1:35636)
0809 09:37:22.862650 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0809 09:37:22.862747 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.188:46547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108}
I20260809 09:37:22.863677 22261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:37:22.863429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.188:46547 (local address 127.0.0.1:35648)
0809 09:37:22.863546 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0809 09:37:22.863603 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.188:46547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260809 09:37:22.863873 22257 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.125.188:46547: Network error: Client connection negotiation failed: client connection to 127.20.125.188:46547: connect: Connection refused (error 111)
W20260809 09:37:22.870024 22394 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 09:37:22.870335 22394 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 09:37:22.870406 22394 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 09:37:22.870507 22394 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 09:37:22.874606 22394 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 09:37:22.874682 22394 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 09:37:22.874727 22394 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 09:37:22.874763 22394 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 09:37:22.874796 22394 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 09:37:22.875089 22382 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.125.188" port: 46547 } attempt: 2
I20260809 09:37:22.875912 22388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:37:22.875618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.188:46547 (local address 127.0.0.1:35652)
0809 09:37:22.875777 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0809 09:37:22.875838 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.188:46547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260809 09:37:22.876607 22388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:37:22.876396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.188:46547 (local address 127.0.0.1:35666)
0809 09:37:22.876500 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0809 09:37:22.876553 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.125.188:46547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260809 09:37:22.876727 22382 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.125.188:46547: Network error: Client connection negotiation failed: client connection to 127.20.125.188:46547: connect: Connection refused (error 111)
I20260809 09:37:22.880385 22394 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.125.148:42931
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786267941944493-20982-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786267941944493-20982-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.20.125.190:43161,127.20.125.189:45207,127.20.125.188:46547
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786267941944493-20982-0/krb5kdc/kudu_127.20.125.188.keytab
--principal=kudu/127.20.125.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.125.188:46547
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786267941944493-20982-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.20.125.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22394
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786267941944493-20982-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 08:38:55 UTC on ccycloud.quasar-njjleu.root.comops.site
build id 4575935
I20260809 09:37:22.881449 22394 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 09:37:22.882447 22394 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 09:37:22.891321 22401 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 09:37:22.891330 22402 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:37:22.891539 22394 server_base.cc:1034] running on GCE node
W20260809 09:37:22.891330 22405 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 09:37:22.892387 22394 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 09:37:22.893565 22394 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 09:37:22.894814 22394 hybrid_clock.cc:648] HybridClock initialized: now 1786268242894794 us; error 37 us; skew 500 ppm
Aug 09 09:37:22 dist-test-slave-7kzw krb5kdc[22129](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268242, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 09:37:22.898238 22394 init.cc:377] Logged in from keytab as kudu/127.20.125.188@KRBTEST.COM (short username kudu)
I20260809 09:37:22.899276 22394 webserver.cc:460] Webserver started at http://127.20.125.188:36401/ using document root <none> and password file <none>
I20260809 09:37:22.899891 22394 fs_manager.cc:359] Metadata directory not provided
I20260809 09:37:22.899957 22394 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 09:37:22.900144 22394 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 09:37:22.902354 22394 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786267941944493-20982-0/minicluster-data/master-2/data/instance:
uuid: "6aacdc8aff804d12b4457251995cc536"
format_stamp: "Formatted at 2026-08-09 09:37:22 on dist-test-slave-7kzw"
server_key: "53a247dce633d02396690433333cf281"
server_key_iv: "4fbdabe7b2aaf300096531eb22a87502"
server_key_version: "encryptionkey@0"
I20260809 09:37:22.903033 22394 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786267941944493-20982-0/minicluster-data/master-2/wal/instance:
uuid: "6aacdc8aff804d12b4457251995cc536"
format_stamp: "Formatted at 2026-08-09 09:37:22 on dist-test-slave-7kzw"
server_key: "53a247dce633d02396690433333cf281"
server_key_iv: "4fbdabe7b2aaf300096531eb22a87502"
server_key_version: "encryptionkey@0"
I20260809 09:37:22.907239 22394 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260809 09:37:22.909978 22423 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 09:37:22.911545 22394 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260809 09:37:22.911711 22394 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786267941944493-20982-0/minicluster-data/master-2/data,/tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786267941944493-20982-0/minicluster-data/master-2/wal
uuid: "6aacdc8aff804d12b4457251995cc536"
format_stamp: "Formatted at 2026-08-09 09:37:22 on dist-test-slave-7kzw"
server_key: "53a247dce633d02396690433333cf281"
server_key_iv: "4fbdabe7b2aaf300096531eb22a87502"
server_key_version: "encryptionkey@0"
I20260809 09:37:22.911863 22394 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786267941944493-20982-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786267941944493-20982-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786267941944493-20982-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 09:37:22.928507 22394 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 09:37:22.932092 22394 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 09:37:22.932271 22394 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 09:37:22.939843 22394 rpc_server.cc:307] RPC server started. Bound to: 127.20.125.188:46547
I20260809 09:37:22.940138 22503 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.125.188:46547 every 8 connection(s)
I20260809 09:37:22.940907 22394 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786267941944493-20982-0/minicluster-data/master-2/data/info.pb
I20260809 09:37:22.943727 22506 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 09:37:22.947535 22506 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.125.190" port: 43161 } has no permanent_uuid. Determining permanent_uuid...
I20260809 09:37:22.950757 20982 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu as pid 22394
I20260809 09:37:22.950902 20982 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYHpNFG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786267941944493-20982-0/minicluster-data/master-2/wal/instance
I20260809 09:37:22.951247 20982 external_mini_cluster.cc:1428] Setting key 79886df6cc19fa09bc432e191916d8ab
Aug 09 09:37:22 dist-test-slave-7kzw krb5kdc[22129](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268242, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.188@KRBTEST.COM for kudu/127.20.125.190@KRBTEST.COM
Aug 09 09:37:22 dist-test-slave-7kzw krb5kdc[22129](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268242, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.125.188@KRBTEST.COM
I20260809 09:37:22.962723 22262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:37:22.949381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48308 (local address 127.20.125.190:43161)
0809 09:37:22.949569 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0809 09:37:22.949575 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 09:37:22.950450 (+ 875us) server_negotiation.cc:407] Connection header received
0809 09:37:22.951698 (+ 1248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:37:22.951703 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:37:22.951778 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:37:22.951889 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:37:22.953361 (+ 1472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:37:22.953870 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:37:22.954923 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:37:22.955159 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:37:22.955274 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:37:22.958566 (+ 3292us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:37:22.958570 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:37:22.958573 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:37:22.958605 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:37:22.960356 (+ 1751us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:37:22.960911 (+ 555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:37:22.960916 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:37:22.960918 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:37:22.961013 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:37:22.961384 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:37:22.961389 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:37:22.961391 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:37:22.961605 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:37:22.961723 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0809 09:37:22.962370 (+ 647us) server_negotiation.cc:299] Negotiation successful
0809 09:37:22.962540 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260809 09:37:22.963056 22257 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.125.188" port: 46547 } attempt: 3
I20260809 09:37:22.963780 22508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:37:22.949826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.190:43161 (local address 127.0.0.1:48308)
0809 09:37:22.950287 (+ 461us) negotiation.cc:107] Waiting for socket to connect
0809 09:37:22.950329 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 09:37:22.951452 (+ 1123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:37:22.952074 (+ 622us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:37:22.952085 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:37:22.952596 (+ 511us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:37:22.953210 (+ 614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:37:22.953225 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:37:22.953983 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:37:22.953989 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:37:22.954798 (+ 809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:37:22.954808 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:37:22.955307 (+ 499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:37:22.955311 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:37:22.955482 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:37:22.956144 (+ 662us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 09:37:22.956161 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:37:22.958374 (+ 2213us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 09:37:22.960475 (+ 2101us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:37:22.960487 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:37:22.960498 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:37:22.960795 (+ 297us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:37:22.961115 (+ 320us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:37:22.961119 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:37:22.961121 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:37:22.961279 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:37:22.961728 (+ 449us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 09:37:22.961764 (+ 36us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 09:37:22.962032 (+ 268us) client_negotiation.cc:769] Sending connection context
0809 09:37:22.962243 (+ 211us) client_negotiation.cc:240] Negotiation successful
0809 09:37:22.962617 (+ 374us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":274,"mutex_wait_us":1,"thread_start_us":126,"threads_started":1}
I20260809 09:37:22.964421 22511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:37:22.952030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35682 (local address 127.20.125.188:46547)
0809 09:37:22.952428 (+ 398us) server_negotiation.cc:206] Beginning negotiation
0809 09:37:22.952440 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 09:37:22.952465 (+ 25us) server_negotiation.cc:407] Connection header received
0809 09:37:22.952506 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:37:22.952511 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:37:22.952734 (+ 223us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:37:22.952894 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:37:22.953712 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:37:22.954696 (+ 984us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:37:22.955606 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:37:22.956165 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:37:22.956292 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:37:22.959732 (+ 3440us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:37:22.959742 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:37:22.959760 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:37:22.959785 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:37:22.961603 (+ 1818us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:37:22.962124 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:37:22.962130 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:37:22.962131 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:37:22.962216 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:37:22.962604 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:37:22.962610 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:37:22.962612 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:37:22.963007 (+ 395us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:37:22.963282 (+ 275us) server_negotiation.cc:1035] Waiting for connection context
0809 09:37:22.964063 (+ 781us) server_negotiation.cc:299] Negotiation successful
0809 09:37:22.964235 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":40,"server-negotiator.queue_time_us":315,"thread_start_us":151,"threads_started":1}
I20260809 09:37:22.965502 22506 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.125.189" port: 45207 } has no permanent_uuid. Determining permanent_uuid...
I20260809 09:37:22.967883 22382 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.125.188" port: 46547 } attempt: 3
Aug 09 09:37:22 dist-test-slave-7kzw krb5kdc[22129](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268242, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.190@KRBTEST.COM for kudu/127.20.125.188@KRBTEST.COM
Aug 09 09:37:22 dist-test-slave-7kzw krb5kdc[22129](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268242, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.188@KRBTEST.COM for kudu/127.20.125.189@KRBTEST.COM
Aug 09 09:37:22 dist-test-slave-7kzw krb5kdc[22129](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268242, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.189@KRBTEST.COM for kudu/127.20.125.188@KRBTEST.COM
I20260809 09:37:22.974144 22261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:37:22.963469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.188:46547 (local address 127.0.0.1:35690)
0809 09:37:22.963644 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0809 09:37:22.963660 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 09:37:22.963740 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:37:22.964242 (+ 502us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:37:22.964249 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:37:22.964440 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:37:22.964572 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:37:22.964581 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:37:22.965744 (+ 1163us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:37:22.965748 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:37:22.966676 (+ 928us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:37:22.966688 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:37:22.967342 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:37:22.967345 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:37:22.967438 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:37:22.967952 (+ 514us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 09:37:22.967970 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:37:22.969715 (+ 1745us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 09:37:22.972182 (+ 2467us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:37:22.972186 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:37:22.972188 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:37:22.972489 (+ 301us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:37:22.972869 (+ 380us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:37:22.972873 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:37:22.972875 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:37:22.972981 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:37:22.973503 (+ 522us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 09:37:22.973507 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 09:37:22.973629 (+ 122us) client_negotiation.cc:769] Sending connection context
0809 09:37:22.973730 (+ 101us) client_negotiation.cc:240] Negotiation successful
0809 09:37:22.973905 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107}
I20260809 09:37:22.974900 22512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:37:22.963577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35690 (local address 127.20.125.188:46547)
0809 09:37:22.963935 (+ 358us) server_negotiation.cc:206] Beginning negotiation
0809 09:37:22.963947 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 09:37:22.963961 (+ 14us) server_negotiation.cc:407] Connection header received
0809 09:37:22.963992 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:37:22.963995 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:37:22.964038 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:37:22.964190 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:37:22.964877 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:37:22.965620 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:37:22.966835 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:37:22.967228 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:37:22.967341 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:37:22.969875 (+ 2534us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:37:22.969879 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:37:22.969882 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:37:22.969906 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:37:22.972048 (+ 2142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:37:22.972652 (+ 604us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:37:22.972656 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:37:22.972658 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:37:22.972753 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:37:22.973114 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:37:22.973118 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:37:22.973121 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:37:22.973378 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:37:22.974366 (+ 988us) server_negotiation.cc:1035] Waiting for connection context
0809 09:37:22.974452 (+ 86us) server_negotiation.cc:299] Negotiation successful
0809 09:37:22.974649 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":268,"thread_start_us":130,"threads_started":1}
Aug 09 09:37:22 dist-test-slave-7kzw krb5kdc[22129](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268242, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.125.190@KRBTEST.COM
I20260809 09:37:22.978138 22388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:37:22.968260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.188:46547 (local address 127.0.0.1:35696)
0809 09:37:22.968502 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0809 09:37:22.968517 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 09:37:22.968587 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:37:22.968877 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:37:22.968881 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:37:22.969080 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:37:22.969174 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:37:22.969181 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:37:22.970568 (+ 1387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:37:22.970571 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:37:22.971387 (+ 816us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:37:22.971398 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:37:22.972061 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:37:22.972066 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:37:22.972152 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:37:22.972645 (+ 493us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 09:37:22.972663 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:37:22.974154 (+ 1491us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 09:37:22.976423 (+ 2269us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:37:22.976430 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:37:22.976434 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:37:22.976679 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:37:22.977085 (+ 406us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:37:22.977089 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:37:22.977091 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:37:22.977190 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:37:22.977642 (+ 452us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 09:37:22.977645 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 09:37:22.977744 (+ 99us) client_negotiation.cc:769] Sending connection context
0809 09:37:22.977849 (+ 105us) client_negotiation.cc:240] Negotiation successful
0809 09:37:22.977987 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":165}
I20260809 09:37:22.978751 22511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:37:22.968589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35696 (local address 127.20.125.188:46547)
0809 09:37:22.968695 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0809 09:37:22.968699 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 09:37:22.968710 (+ 11us) server_negotiation.cc:407] Connection header received
0809 09:37:22.968744 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:37:22.968746 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:37:22.968775 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:37:22.968856 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:37:22.969305 (+ 449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:37:22.970414 (+ 1109us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:37:22.971534 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:37:22.971911 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:37:22.972369 (+ 458us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:37:22.974309 (+ 1940us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:37:22.974313 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:37:22.974317 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:37:22.974345 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:37:22.976299 (+ 1954us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:37:22.976802 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:37:22.976807 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:37:22.976809 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:37:22.976910 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:37:22.977296 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:37:22.977300 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:37:22.977302 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:37:22.977520 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:37:22.977629 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0809 09:37:22.978486 (+ 857us) server_negotiation.cc:299] Negotiation successful
0809 09:37:22.978598 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260809 09:37:22.979161 22257 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d78be03a73df4aba99e1df24a60ad797: Bootstrap starting.
I20260809 09:37:22.981169 22262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:37:22.967528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48314 (local address 127.20.125.190:43161)
0809 09:37:22.967970 (+ 442us) server_negotiation.cc:206] Beginning negotiation
0809 09:37:22.967975 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 09:37:22.968072 (+ 97us) server_negotiation.cc:407] Connection header received
0809 09:37:22.968114 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:37:22.968119 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:37:22.968169 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:37:22.968251 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:37:22.969213 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:37:22.970058 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:37:22.972182 (+ 2124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:37:22.972911 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:37:22.972994 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:37:22.977541 (+ 4547us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:37:22.977547 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:37:22.977554 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:37:22.977588 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:37:22.979525 (+ 1937us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:37:22.979992 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:37:22.979995 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:37:22.979996 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:37:22.980076 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:37:22.980383 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:37:22.980387 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:37:22.980388 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:37:22.980606 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:37:22.980756 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0809 09:37:22.980963 (+ 207us) server_negotiation.cc:299] Negotiation successful
0809 09:37:22.981067 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260809 09:37:22.982050 22257 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d78be03a73df4aba99e1df24a60ad797: Neither blocks nor log segments found. Creating new log.
I20260809 09:37:22.983404 22257 log.cc:824] T 00000000000000000000000000000000 P d78be03a73df4aba99e1df24a60ad797: Log is configured to *not* fsync() on all Append() calls
I20260809 09:37:22.984473 22392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:37:22.966686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43712 (local address 127.20.125.189:45207)
0809 09:37:22.966929 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0809 09:37:22.966934 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 09:37:22.966949 (+ 15us) server_negotiation.cc:407] Connection header received
0809 09:37:22.966990 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:37:22.966994 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:37:22.967046 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:37:22.967136 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:37:22.967663 (+ 527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:37:22.968445 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:37:22.969939 (+ 1494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:37:22.970321 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:37:22.970393 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:37:22.972789 (+ 2396us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:37:22.972794 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:37:22.972798 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:37:22.972822 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:37:22.974768 (+ 1946us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:37:22.975322 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:37:22.975325 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:37:22.975327 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:37:22.975416 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:37:22.975745 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:37:22.975752 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:37:22.975756 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:37:22.976054 (+ 298us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:37:22.976151 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0809 09:37:22.984213 (+ 8062us) server_negotiation.cc:299] Negotiation successful
0809 09:37:22.984333 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139}
I20260809 09:37:22.985888 22382 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 222dee7fe32c4ccaa54c34d7a072c9da: Bootstrap starting.
I20260809 09:37:22.986081 22257 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d78be03a73df4aba99e1df24a60ad797: No bootstrap required, opened a new log
Aug 09 09:37:22 dist-test-slave-7kzw krb5kdc[22129](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268242, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.125.189@KRBTEST.COM
I20260809 09:37:22.988523 22382 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 222dee7fe32c4ccaa54c34d7a072c9da: Neither blocks nor log segments found. Creating new log.
I20260809 09:37:22.989466 22382 log.cc:824] T 00000000000000000000000000000000 P 222dee7fe32c4ccaa54c34d7a072c9da: Log is configured to *not* fsync() on all Append() calls
I20260809 09:37:22.990417 22257 raft_consensus.cc:348] T 00000000000000000000000000000000 P d78be03a73df4aba99e1df24a60ad797 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d78be03a73df4aba99e1df24a60ad797" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 43161 } } peers { permanent_uuid: "222dee7fe32c4ccaa54c34d7a072c9da" member_type: VOTER last_known_addr { host: "127.20.125.189" port: 45207 } } peers { permanent_uuid: "6aacdc8aff804d12b4457251995cc536" member_type: VOTER last_known_addr { host: "127.20.125.188" port: 46547 } }
I20260809 09:37:22.990727 22257 raft_consensus.cc:729] T 00000000000000000000000000000000 P d78be03a73df4aba99e1df24a60ad797 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d78be03a73df4aba99e1df24a60ad797, State: Initialized, Role: FOLLOWER
I20260809 09:37:22.990958 22508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:37:22.966032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.189:45207 (local address 127.0.0.1:43712)
0809 09:37:22.966179 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0809 09:37:22.966192 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 09:37:22.966252 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:37:22.967200 (+ 948us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:37:22.967204 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:37:22.967389 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:37:22.967504 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:37:22.967511 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:37:22.968964 (+ 1453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:37:22.968967 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:37:22.969802 (+ 835us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:37:22.969812 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:37:22.970596 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:37:22.970599 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:37:22.970695 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:37:22.971112 (+ 417us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 09:37:22.971126 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:37:22.972631 (+ 1505us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 09:37:22.974885 (+ 2254us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:37:22.974888 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:37:22.974890 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:37:22.975206 (+ 316us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:37:22.975527 (+ 321us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:37:22.975530 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:37:22.975532 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:37:22.975626 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:37:22.983912 (+ 8286us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 09:37:22.983915 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 09:37:22.984009 (+ 94us) client_negotiation.cc:769] Sending connection context
0809 09:37:22.990526 (+ 6517us) client_negotiation.cc:240] Negotiation successful
0809 09:37:22.990741 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":93}
I20260809 09:37:22.991075 22257 consensus_queue.cc:260] T 00000000000000000000000000000000 P d78be03a73df4aba99e1df24a60ad797 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d78be03a73df4aba99e1df24a60ad797" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 43161 } } peers { permanent_uuid: "222dee7fe32c4ccaa54c34d7a072c9da" member_type: VOTER last_known_addr { host: "127.20.125.189" port: 45207 } } peers { permanent_uuid: "6aacdc8aff804d12b4457251995cc536" member_type: VOTER last_known_addr { host: "127.20.125.188" port: 46547 } }
I20260809 09:37:22.991575 22515 sys_catalog.cc:455] T 00000000000000000000000000000000 P d78be03a73df4aba99e1df24a60ad797 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d78be03a73df4aba99e1df24a60ad797" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 43161 } } peers { permanent_uuid: "222dee7fe32c4ccaa54c34d7a072c9da" member_type: VOTER last_known_addr { host: "127.20.125.189" port: 45207 } } peers { permanent_uuid: "6aacdc8aff804d12b4457251995cc536" member_type: VOTER last_known_addr { host: "127.20.125.188" port: 46547 } } }
I20260809 09:37:22.991756 22515 sys_catalog.cc:458] T 00000000000000000000000000000000 P d78be03a73df4aba99e1df24a60ad797 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 09:37:22.991822 22382 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 222dee7fe32c4ccaa54c34d7a072c9da: No bootstrap required, opened a new log
I20260809 09:37:22.991978 22506 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.125.188" port: 46547 } has no permanent_uuid. Determining permanent_uuid...
I20260809 09:37:22.992517 22257 sys_catalog.cc:564] T 00000000000000000000000000000000 P d78be03a73df4aba99e1df24a60ad797 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 09:37:22.993551 22391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:37:22.982637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43726 (local address 127.20.125.189:45207)
0809 09:37:22.982764 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0809 09:37:22.982769 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 09:37:22.982828 (+ 59us) server_negotiation.cc:407] Connection header received
0809 09:37:22.982980 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:37:22.982983 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:37:22.983030 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:37:22.983143 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:37:22.983778 (+ 635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:37:22.984678 (+ 900us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:37:22.985517 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:37:22.985819 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:37:22.985941 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:37:22.988565 (+ 2624us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:37:22.988569 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:37:22.988572 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:37:22.988602 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:37:22.990313 (+ 1711us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:37:22.990904 (+ 591us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:37:22.990907 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:37:22.990908 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:37:22.991006 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:37:22.992097 (+ 1091us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:37:22.992100 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:37:22.992102 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:37:22.992339 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:37:22.993166 (+ 827us) server_negotiation.cc:1035] Waiting for connection context
0809 09:37:22.993234 (+ 68us) server_negotiation.cc:299] Negotiation successful
0809 09:37:22.993372 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260809 09:37:22.996812 22382 raft_consensus.cc:348] T 00000000000000000000000000000000 P 222dee7fe32c4ccaa54c34d7a072c9da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d78be03a73df4aba99e1df24a60ad797" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 43161 } } peers { permanent_uuid: "222dee7fe32c4ccaa54c34d7a072c9da" member_type: VOTER last_known_addr { host: "127.20.125.189" port: 45207 } } peers { permanent_uuid: "6aacdc8aff804d12b4457251995cc536" member_type: VOTER last_known_addr { host: "127.20.125.188" port: 46547 } }
I20260809 09:37:22.997044 22382 raft_consensus.cc:729] T 00000000000000000000000000000000 P 222dee7fe32c4ccaa54c34d7a072c9da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 222dee7fe32c4ccaa54c34d7a072c9da, State: Initialized, Role: FOLLOWER
I20260809 09:37:22.997332 20982 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 09:37:22.997380 22382 consensus_queue.cc:260] T 00000000000000000000000000000000 P 222dee7fe32c4ccaa54c34d7a072c9da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d78be03a73df4aba99e1df24a60ad797" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 43161 } } peers { permanent_uuid: "222dee7fe32c4ccaa54c34d7a072c9da" member_type: VOTER last_known_addr { host: "127.20.125.189" port: 45207 } } peers { permanent_uuid: "6aacdc8aff804d12b4457251995cc536" member_type: VOTER last_known_addr { host: "127.20.125.188" port: 46547 } }
W20260809 09:37:22.998426 22530 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d78be03a73df4aba99e1df24a60ad797: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 09:37:22.998526 22530 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 09:37:22.998857 22219 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "d78be03a73df4aba99e1df24a60ad797"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48314
I20260809 09:37:22.999071 22219 raft_consensus.cc:482] T 00000000000000000000000000000000 P d78be03a73df4aba99e1df24a60ad797 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 09:37:22.999164 22219 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d78be03a73df4aba99e1df24a60ad797 [term 0 FOLLOWER]: Advancing to term 1
I20260809 09:37:23.000550 22219 raft_consensus.cc:504] T 00000000000000000000000000000000 P d78be03a73df4aba99e1df24a60ad797 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d78be03a73df4aba99e1df24a60ad797" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 43161 } } peers { permanent_uuid: "222dee7fe32c4ccaa54c34d7a072c9da" member_type: VOTER last_known_addr { host: "127.20.125.189" port: 45207 } } peers { permanent_uuid: "6aacdc8aff804d12b4457251995cc536" member_type: VOTER last_known_addr { host: "127.20.125.188" port: 46547 } }
I20260809 09:37:22.997782 22149 master_runner.cc:186] Error getting master registration for 127.20.125.189:45207: OK, instance_id { permanent_uuid: "222dee7fe32c4ccaa54c34d7a072c9da" instance_seqno: 1786268242710428 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 09:37:23.001160 22149 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 09:37:23.001575 22219 leader_election.cc:288] T 00000000000000000000000000000000 P d78be03a73df4aba99e1df24a60ad797 [CANDIDATE]: Term 1 election: Requested vote from peers 222dee7fe32c4ccaa54c34d7a072c9da (127.20.125.189:45207), 6aacdc8aff804d12b4457251995cc536 (127.20.125.188:46547)
I20260809 09:37:23.002978 22533 sys_catalog.cc:455] T 00000000000000000000000000000000 P 222dee7fe32c4ccaa54c34d7a072c9da [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d78be03a73df4aba99e1df24a60ad797" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 43161 } } peers { permanent_uuid: "222dee7fe32c4ccaa54c34d7a072c9da" member_type: VOTER last_known_addr { host: "127.20.125.189" port: 45207 } } peers { permanent_uuid: "6aacdc8aff804d12b4457251995cc536" member_type: VOTER last_known_addr { host: "127.20.125.188" port: 46547 } } }
I20260809 09:37:23.003130 22533 sys_catalog.cc:458] T 00000000000000000000000000000000 P 222dee7fe32c4ccaa54c34d7a072c9da [sys.catalog]: This master's current role is: FOLLOWER
Aug 09 09:37:23 dist-test-slave-7kzw krb5kdc[22129](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786268242, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.125.188@KRBTEST.COM for kudu/127.20.125.188@KRBTEST.COM
I20260809 09:37:23.003855 22340 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d78be03a73df4aba99e1df24a60ad797" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "222dee7fe32c4ccaa54c34d7a072c9da"
I20260809 09:37:23.004123 22340 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 222dee7fe32c4ccaa54c34d7a072c9da [term 0 FOLLOWER]: Advancing to term 1
I20260809 09:37:23.003600 22382 sys_catalog.cc:564] T 00000000000000000000000000000000 P 222dee7fe32c4ccaa54c34d7a072c9da [sys.catalog]: configured and running, proceeding with master startup.
I20260809 09:37:23.005908 22340 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 222dee7fe32c4ccaa54c34d7a072c9da [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d78be03a73df4aba99e1df24a60ad797 in term 1.
I20260809 09:37:23.006412 22181 leader_election.cc:302] T 00000000000000000000000000000000 P d78be03a73df4aba99e1df24a60ad797 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 222dee7fe32c4ccaa54c34d7a072c9da, d78be03a73df4aba99e1df24a60ad797; no voters:
I20260809 09:37:23.006840 22515 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d78be03a73df4aba99e1df24a60ad797 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 09:37:23.007086 22515 raft_consensus.cc:686] T 00000000000000000000000000000000 P d78be03a73df4aba99e1df24a60ad797 [term 1 LEADER]: Becoming Leader. State: Replica: d78be03a73df4aba99e1df24a60ad797, State: Running, Role: LEADER
I20260809 09:37:23.007495 22515 consensus_queue.cc:237] T 00000000000000000000000000000000 P d78be03a73df4aba99e1df24a60ad797 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d78be03a73df4aba99e1df24a60ad797" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 43161 } } peers { permanent_uuid: "222dee7fe32c4ccaa54c34d7a072c9da" member_type: VOTER last_known_addr { host: "127.20.125.189" port: 45207 } } peers { permanent_uuid: "6aacdc8aff804d12b4457251995cc536" member_type: VOTER last_known_addr { host: "127.20.125.188" port: 46547 } }
I20260809 09:37:23.007700 22508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:37:22.992491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.125.188:46547 (local address 127.0.0.1:35700)
0809 09:37:22.992701 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0809 09:37:22.992715 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 09:37:22.992788 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 09:37:22.998850 (+ 6062us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 09:37:22.998854 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 09:37:22.999094 (+ 240us) client_negotiation.cc:189] Negotiated authn=SASL
0809 09:37:22.999249 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:37:22.999260 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:37:23.000311 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:37:23.000316 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:37:23.001064 (+ 748us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 09:37:23.001076 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 09:37:23.001646 (+ 570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 09:37:23.001650 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 09:37:23.001727 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:37:23.002270 (+ 543us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 09:37:23.002291 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 09:37:23.003960 (+ 1669us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 09:37:23.006149 (+ 2189us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:37:23.006152 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:37:23.006154 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:37:23.006398 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:37:23.006711 (+ 313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 09:37:23.006714 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 09:37:23.006716 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 09:37:23.006794 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 09:37:23.007203 (+ 409us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 09:37:23.007206 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 09:37:23.007308 (+ 102us) client_negotiation.cc:769] Sending connection context
0809 09:37:23.007390 (+ 82us) client_negotiation.cc:240] Negotiation successful
0809 09:37:23.007530 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145}
I20260809 09:37:23.008224 22511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:37:22.992969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35700 (local address 127.20.125.188:46547)
0809 09:37:22.998577 (+ 5608us) server_negotiation.cc:206] Beginning negotiation
0809 09:37:22.998584 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 09:37:22.998604 (+ 20us) server_negotiation.cc:407] Connection header received
0809 09:37:22.998647 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:37:22.998651 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:37:22.998700 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:37:22.999364 (+ 664us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:37:22.999429 (+ 65us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:37:23.000182 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:37:23.001210 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:37:23.001524 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:37:23.001973 (+ 449us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:37:23.004155 (+ 2182us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:37:23.004159 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:37:23.004162 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:37:23.004187 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:37:23.006018 (+ 1831us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:37:23.006529 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:37:23.006532 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:37:23.006533 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:37:23.006616 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:37:23.006888 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:37:23.006891 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:37:23.006893 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:37:23.007101 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:37:23.007868 (+ 767us) server_negotiation.cc:1035] Waiting for connection context
0809 09:37:23.007948 (+ 80us) server_negotiation.cc:299] Negotiation successful
0809 09:37:23.008075 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5522}
I20260809 09:37:23.009642 22515 sys_catalog.cc:455] T 00000000000000000000000000000000 P d78be03a73df4aba99e1df24a60ad797 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d78be03a73df4aba99e1df24a60ad797. Latest consensus state: current_term: 1 leader_uuid: "d78be03a73df4aba99e1df24a60ad797" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d78be03a73df4aba99e1df24a60ad797" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 43161 } } peers { permanent_uuid: "222dee7fe32c4ccaa54c34d7a072c9da" member_type: VOTER last_known_addr { host: "127.20.125.189" port: 45207 } } peers { permanent_uuid: "6aacdc8aff804d12b4457251995cc536" member_type: VOTER last_known_addr { host: "127.20.125.188" port: 46547 } } }
I20260809 09:37:23.009768 22515 sys_catalog.cc:458] T 00000000000000000000000000000000 P d78be03a73df4aba99e1df24a60ad797 [sys.catalog]: This master's current role is: LEADER
I20260809 09:37:23.011679 22506 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6aacdc8aff804d12b4457251995cc536: Bootstrap starting.
I20260809 09:37:23.014613 22506 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6aacdc8aff804d12b4457251995cc536: Neither blocks nor log segments found. Creating new log.
I20260809 09:37:23.016100 22506 log.cc:824] T 00000000000000000000000000000000 P 6aacdc8aff804d12b4457251995cc536: Log is configured to *not* fsync() on all Append() calls
I20260809 09:37:23.016867 22559 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 09:37:23.018596 22559 catalog_manager.cc:1484] Initializing Kudu cluster ID...
W20260809 09:37:23.025044 22560 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 222dee7fe32c4ccaa54c34d7a072c9da: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 09:37:23.025605 22560 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 09:37:23.022991 22268 master_runner.cc:186] Error getting master registration for 127.20.125.188:46547: OK, instance_id { permanent_uuid: "6aacdc8aff804d12b4457251995cc536" instance_seqno: 1786268242937646 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 09:37:23.027680 22268 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 09:37:23.032232 22262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:37:23.006877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48324 (local address 127.20.125.190:43161)
0809 09:37:23.006995 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0809 09:37:23.007002 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 09:37:23.011251 (+ 4249us) server_negotiation.cc:407] Connection header received
0809 09:37:23.011362 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:37:23.011366 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:37:23.011418 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:37:23.011515 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:37:23.012496 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:37:23.013299 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:37:23.014895 (+ 1596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:37:23.015246 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:37:23.015347 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:37:23.027619 (+ 12272us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:37:23.027624 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:37:23.027627 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:37:23.027664 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:37:23.029582 (+ 1918us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:37:23.030418 (+ 836us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:37:23.030423 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:37:23.030425 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:37:23.030593 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:37:23.030924 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:37:23.030927 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:37:23.030929 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:37:23.031157 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:37:23.031279 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0809 09:37:23.031646 (+ 367us) server_negotiation.cc:299] Negotiation successful
0809 09:37:23.032058 (+ 412us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260809 09:37:23.035298 22391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:37:23.014673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43732 (local address 127.20.125.189:45207)
0809 09:37:23.014822 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0809 09:37:23.014828 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 09:37:23.015985 (+ 1157us) server_negotiation.cc:407] Connection header received
0809 09:37:23.016425 (+ 440us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:37:23.016429 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:37:23.016484 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:37:23.016575 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:37:23.022455 (+ 5880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:37:23.023277 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:37:23.026542 (+ 3265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:37:23.026911 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:37:23.027003 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:37:23.031385 (+ 4382us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:37:23.031393 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:37:23.031397 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:37:23.031430 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:37:23.033302 (+ 1872us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:37:23.033804 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:37:23.033808 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:37:23.033810 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:37:23.033910 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:37:23.034229 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:37:23.034232 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:37:23.034234 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:37:23.034444 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:37:23.034611 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0809 09:37:23.034991 (+ 380us) server_negotiation.cc:299] Negotiation successful
0809 09:37:23.035136 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260809 09:37:23.036010 22506 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6aacdc8aff804d12b4457251995cc536: No bootstrap required, opened a new log
I20260809 09:37:23.039317 22512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:37:23.006265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35716 (local address 127.20.125.188:46547)
0809 09:37:23.006386 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0809 09:37:23.006391 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 09:37:23.008149 (+ 1758us) server_negotiation.cc:407] Connection header received
0809 09:37:23.015645 (+ 7496us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:37:23.015648 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:37:23.015700 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:37:23.015786 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:37:23.017752 (+ 1966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:37:23.018678 (+ 926us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:37:23.019833 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:37:23.020204 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:37:23.020293 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:37:23.029255 (+ 8962us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 09:37:23.029260 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 09:37:23.029263 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 09:37:23.029297 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 09:37:23.031297 (+ 2000us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:37:23.032177 (+ 880us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:37:23.032181 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:37:23.032184 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:37:23.032308 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 09:37:23.032911 (+ 603us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 09:37:23.032917 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 09:37:23.032919 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 09:37:23.033183 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 09:37:23.033335 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0809 09:37:23.038992 (+ 5657us) server_negotiation.cc:299] Negotiation successful
0809 09:37:23.039140 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260809 09:37:23.039867 22340 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 222dee7fe32c4ccaa54c34d7a072c9da [term 1 FOLLOWER]: Refusing update from remote peer d78be03a73df4aba99e1df24a60ad797: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 09:37:23.040548 22515 consensus_queue.cc:1035] T 00000000000000000000000000000000 P d78be03a73df4aba99e1df24a60ad797 [LEADER]: Connected to new peer: Peer: permanent_uuid: "222dee7fe32c4ccaa54c34d7a072c9da" member_type: VOTER last_known_addr { host: "127.20.125.189" port: 45207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 09:37:23.040757 22506 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6aacdc8aff804d12b4457251995cc536 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d78be03a73df4aba99e1df24a60ad797" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 43161 } } peers { permanent_uuid: "222dee7fe32c4ccaa54c34d7a072c9da" member_type: VOTER last_known_addr { host: "127.20.125.189" port: 45207 } } peers { permanent_uuid: "6aacdc8aff804d12b4457251995cc536" member_type: VOTER last_known_addr { host: "127.20.125.188" port: 46547 } }
I20260809 09:37:23.041025 22506 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6aacdc8aff804d12b4457251995cc536 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6aacdc8aff804d12b4457251995cc536, State: Initialized, Role: FOLLOWER
I20260809 09:37:23.041411 22506 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6aacdc8aff804d12b4457251995cc536 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d78be03a73df4aba99e1df24a60ad797" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 43161 } } peers { permanent_uuid: "222dee7fe32c4ccaa54c34d7a072c9da" member_type: VOTER last_known_addr { host: "127.20.125.189" port: 45207 } } peers { permanent_uuid: "6aacdc8aff804d12b4457251995cc536" member_type: VOTER last_known_addr { host: "127.20.125.188" port: 46547 } }
I20260809 09:37:23.043027 22506 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6aacdc8aff804d12b4457251995cc536 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 09:37:23.043241 22463 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6aacdc8aff804d12b4457251995cc536 [term 0 FOLLOWER]: Advancing to term 1
I20260809 09:37:23.044814 22463 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 6aacdc8aff804d12b4457251995cc536 [term 1 FOLLOWER]: Refusing update from remote peer d78be03a73df4aba99e1df24a60ad797: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 09:37:23.045821 22542 consensus_queue.cc:1035] T 00000000000000000000000000000000 P d78be03a73df4aba99e1df24a60ad797 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6aacdc8aff804d12b4457251995cc536" member_type: VOTER last_known_addr { host: "127.20.125.188" port: 46547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 09:37:23.045338 22464 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d78be03a73df4aba99e1df24a60ad797" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "6aacdc8aff804d12b4457251995cc536"
I20260809 09:37:23.047860 22464 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 6aacdc8aff804d12b4457251995cc536 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d78be03a73df4aba99e1df24a60ad797 in term 1.
W20260809 09:37:23.050087 22587 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 6aacdc8aff804d12b4457251995cc536: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 09:37:23.050169 22587 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 09:37:23.054924 22533 sys_catalog.cc:455] T 00000000000000000000000000000000 P 222dee7fe32c4ccaa54c34d7a072c9da [sys.catalog]: SysCatalogTable state changed. Reason: New leader d78be03a73df4aba99e1df24a60ad797. Latest consensus state: current_term: 1 leader_uuid: "d78be03a73df4aba99e1df24a60ad797" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d78be03a73df4aba99e1df24a60ad797" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 43161 } } peers { permanent_uuid: "222dee7fe32c4ccaa54c34d7a072c9da" member_type: VOTER last_known_addr { host: "127.20.125.189" port: 45207 } } peers { permanent_uuid: "6aacdc8aff804d12b4457251995cc536" member_type: VOTER last_known_addr { host: "127.20.125.188" port: 46547 } } }
I20260809 09:37:23.055060 22533 sys_catalog.cc:458] T 00000000000000000000000000000000 P 222dee7fe32c4ccaa54c34d7a072c9da [sys.catalog]: This master's current role is: FOLLOWER
I20260809 09:37:23.060447 22568 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6aacdc8aff804d12b4457251995cc536 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d78be03a73df4aba99e1df24a60ad797" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d78be03a73df4aba99e1df24a60ad797" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 43161 } } peers { permanent_uuid: "222dee7fe32c4ccaa54c34d7a072c9da" member_type: VOTER last_known_addr { host: "127.20.125.189" port: 45207 } } peers { permanent_uuid: "6aacdc8aff804d12b4457251995cc536" member_type: VOTER last_known_addr { host: "127.20.125.188" port: 46547 } } }
I20260809 09:37:23.060623 22568 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6aacdc8aff804d12b4457251995cc536 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 09:37:23.060745 22568 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6aacdc8aff804d12b4457251995cc536 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d78be03a73df4aba99e1df24a60ad797. Latest consensus state: current_term: 1 leader_uuid: "d78be03a73df4aba99e1df24a60ad797" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d78be03a73df4aba99e1df24a60ad797" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 43161 } } peers { permanent_uuid: "222dee7fe32c4ccaa54c34d7a072c9da" member_type: VOTER last_known_addr { host: "127.20.125.189" port: 45207 } } peers { permanent_uuid: "6aacdc8aff804d12b4457251995cc536" member_type: VOTER last_known_addr { host: "127.20.125.188" port: 46547 } } }
I20260809 09:37:23.060850 22568 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6aacdc8aff804d12b4457251995cc536 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 09:37:23.064404 22542 sys_catalog.cc:455] T 00000000000000000000000000000000 P d78be03a73df4aba99e1df24a60ad797 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d78be03a73df4aba99e1df24a60ad797" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d78be03a73df4aba99e1df24a60ad797" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 43161 } } peers { permanent_uuid: "222dee7fe32c4ccaa54c34d7a072c9da" member_type: VOTER last_known_addr { host: "127.20.125.189" port: 45207 } } peers { permanent_uuid: "6aacdc8aff804d12b4457251995cc536" member_type: VOTER last_known_addr { host: "127.20.125.188" port: 46547 } } }
I20260809 09:37:23.064544 22542 sys_catalog.cc:458] T 00000000000000000000000000000000 P d78be03a73df4aba99e1df24a60ad797 [sys.catalog]: This master's current role is: LEADER
I20260809 09:37:23.064911 22567 mvcc.cc:205] Tried to move back new op lower bound from 7316554723477336064 to 7316554723363016704. Current Snapshot: MvccSnapshot[applied={T|T < 7316554723477336064}]
I20260809 09:37:23.066813 22592 mvcc.cc:205] Tried to move back new op lower bound from 7316554723477336064 to 7316554723363016704. Current Snapshot: MvccSnapshot[applied={T|T < 7316554723477336064}]
I20260809 09:37:23.066994 22590 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6aacdc8aff804d12b4457251995cc536 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d78be03a73df4aba99e1df24a60ad797" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d78be03a73df4aba99e1df24a60ad797" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 43161 } } peers { permanent_uuid: "222dee7fe32c4ccaa54c34d7a072c9da" member_type: VOTER last_known_addr { host: "127.20.125.189" port: 45207 } } peers { permanent_uuid: "6aacdc8aff804d12b4457251995cc536" member_type: VOTER last_known_addr { host: "127.20.125.188" port: 46547 } } }
I20260809 09:37:23.067132 22590 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6aacdc8aff804d12b4457251995cc536 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 09:37:23.067407 22515 sys_catalog.cc:455] T 00000000000000000000000000000000 P d78be03a73df4aba99e1df24a60ad797 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d78be03a73df4aba99e1df24a60ad797" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d78be03a73df4aba99e1df24a60ad797" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 43161 } } peers { permanent_uuid: "222dee7fe32c4ccaa54c34d7a072c9da" member_type: VOTER last_known_addr { host: "127.20.125.189" port: 45207 } } peers { permanent_uuid: "6aacdc8aff804d12b4457251995cc536" member_type: VOTER last_known_addr { host: "127.20.125.188" port: 46547 } } }
I20260809 09:37:23.067543 22515 sys_catalog.cc:458] T 00000000000000000000000000000000 P d78be03a73df4aba99e1df24a60ad797 [sys.catalog]: This master's current role is: LEADER
I20260809 09:37:23.068645 22572 mvcc.cc:205] Tried to move back new op lower bound from 7316554723477336064 to 7316554723363016704. Current Snapshot: MvccSnapshot[applied={T|T < 7316554723477336064}]
I20260809 09:37:23.068820 22533 sys_catalog.cc:455] T 00000000000000000000000000000000 P 222dee7fe32c4ccaa54c34d7a072c9da [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d78be03a73df4aba99e1df24a60ad797" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d78be03a73df4aba99e1df24a60ad797" member_type: VOTER last_known_addr { host: "127.20.125.190" port: 43161 } } peers { permanent_uuid: "222dee7fe32c4ccaa54c34d7a072c9da" member_type: VOTER last_known_addr { host: "127.20.125.189" port: 45207 } } peers { permanent_uuid: "6aacdc8aff804d12b4457251995cc536" member_type: VOTER last_known_addr { host: "127.20.125.188" port: 46547 } } }
I20260809 09:37:23.068925 22533 sys_catalog.cc:458] T 00000000000000000000000000000000 P 222dee7fe32c4ccaa54c34d7a072c9da [sys.catalog]: This master's current role is: FOLLOWER
I20260809 09:37:23.073345 22559 catalog_manager.cc:1347] Generated new cluster ID: bd906f270139445cbc656d95fa868aac
I20260809 09:37:23.073426 22559 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 09:37:23.104210 22559 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 09:37:23.105626 22559 catalog_manager.cc:1504] Loading token signing keys...
I20260809 09:37:23.120592 22559 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d78be03a73df4aba99e1df24a60ad797: Generated new TSK 0
I20260809 09:37:23.121469 22559 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260809 09:37:23.161582 22512 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 09:37:23.158194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35728 (local address 127.20.125.188:46547)
0809 09:37:23.158330 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0809 09:37:23.158335 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 09:37:23.160401 (+ 2066us) server_negotiation.cc:407] Connection header received
0809 09:37:23.160619 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:37:23.160622 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:37:23.160679 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:37:23.160736 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:37:23.161436 (+ 700us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35728: BlockingRecv error: recv got EOF from 127.0.0.1:35728 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
W20260809 09:37:23.163214 22391 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 09:37:23.159084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43736 (local address 127.20.125.189:45207)
0809 09:37:23.159210 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0809 09:37:23.159214 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 09:37:23.162200 (+ 2986us) server_negotiation.cc:407] Connection header received
0809 09:37:23.162379 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:37:23.162383 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:37:23.162433 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:37:23.162509 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:37:23.163080 (+ 571us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43736: BlockingRecv error: recv got EOF from 127.0.0.1:43736 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
W20260809 09:37:23.164904 22262 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 09:37:23.159758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48340 (local address 127.20.125.190:43161)
0809 09:37:23.159874 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0809 09:37:23.159878 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 09:37:23.163639 (+ 3761us) server_negotiation.cc:407] Connection header received
0809 09:37:23.164123 (+ 484us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:37:23.164137 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:37:23.164189 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:37:23.164244 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:37:23.164779 (+ 535us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48340: BlockingRecv error: recv got EOF from 127.0.0.1:48340 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
W20260809 09:37:23.169363 22391 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 09:37:23.167986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43744 (local address 127.20.125.189:45207)
0809 09:37:23.168112 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0809 09:37:23.168116 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 09:37:23.168354 (+ 238us) server_negotiation.cc:407] Connection header received
0809 09:37:23.168572 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:37:23.168576 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:37:23.168654 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:37:23.168713 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:37:23.169256 (+ 543us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43744: BlockingRecv error: recv got EOF from 127.0.0.1:43744 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
W20260809 09:37:23.174629 22391 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 09:37:23.173022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43760 (local address 127.20.125.189:45207)
0809 09:37:23.173123 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0809 09:37:23.173138 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 09:37:23.173622 (+ 484us) server_negotiation.cc:407] Connection header received
0809 09:37:23.173802 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:37:23.173805 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:37:23.173851 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:37:23.173898 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:37:23.174424 (+ 526us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43760: BlockingRecv error: recv got EOF from 127.0.0.1:43760 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
W20260809 09:37:23.180390 22391 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 09:37:23.178909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43762 (local address 127.20.125.189:45207)
0809 09:37:23.179014 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0809 09:37:23.179018 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 09:37:23.179505 (+ 487us) server_negotiation.cc:407] Connection header received
0809 09:37:23.179690 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:37:23.179693 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:37:23.179747 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:37:23.179797 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:37:23.180295 (+ 498us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43762: BlockingRecv error: recv got EOF from 127.0.0.1:43762 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
W20260809 09:37:23.188618 22391 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 09:37:23.186827 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43776 (local address 127.20.125.189:45207)
0809 09:37:23.186970 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0809 09:37:23.186974 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 09:37:23.187552 (+ 578us) server_negotiation.cc:407] Connection header received
0809 09:37:23.187765 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:37:23.187768 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:37:23.187822 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:37:23.187877 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:37:23.188468 (+ 591us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43776: BlockingRecv error: recv got EOF from 127.0.0.1:43776 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
W20260809 09:37:23.200891 22391 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 09:37:23.199272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43790 (local address 127.20.125.189:45207)
0809 09:37:23.199388 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0809 09:37:23.199393 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 09:37:23.199930 (+ 537us) server_negotiation.cc:407] Connection header received
0809 09:37:23.200123 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:37:23.200126 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:37:23.200189 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:37:23.200240 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:37:23.200773 (+ 533us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43790: BlockingRecv error: recv got EOF from 127.0.0.1:43790 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
W20260809 09:37:23.221498 22391 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 09:37:23.219650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43794 (local address 127.20.125.189:45207)
0809 09:37:23.219794 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0809 09:37:23.219799 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 09:37:23.220455 (+ 656us) server_negotiation.cc:407] Connection header received
0809 09:37:23.220676 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:37:23.220680 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:37:23.220736 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:37:23.220789 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:37:23.221376 (+ 587us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43794: BlockingRecv error: recv got EOF from 127.0.0.1:43794 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
W20260809 09:37:23.257477 22391 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 09:37:23.256119 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43796 (local address 127.20.125.189:45207)
0809 09:37:23.256257 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0809 09:37:23.256262 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 09:37:23.256518 (+ 256us) server_negotiation.cc:407] Connection header received
0809 09:37:23.256702 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:37:23.256706 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:37:23.256764 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:37:23.256854 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:37:23.257340 (+ 486us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43796: BlockingRecv error: recv got EOF from 127.0.0.1:43796 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
W20260809 09:37:23.325202 22391 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 09:37:23.323879 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51008 (local address 127.20.125.189:45207)
0809 09:37:23.324019 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0809 09:37:23.324024 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 09:37:23.324223 (+ 199us) server_negotiation.cc:407] Connection header received
0809 09:37:23.324402 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:37:23.324410 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:37:23.324463 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:37:23.324555 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:37:23.325054 (+ 499us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51008: BlockingRecv error: recv got EOF from 127.0.0.1:51008 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
W20260809 09:37:23.456873 22391 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 09:37:23.455605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51012 (local address 127.20.125.189:45207)
0809 09:37:23.455778 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0809 09:37:23.455784 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 09:37:23.455922 (+ 138us) server_negotiation.cc:407] Connection header received
0809 09:37:23.456110 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:37:23.456115 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:37:23.456175 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:37:23.456266 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:37:23.456745 (+ 479us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51012: BlockingRecv error: recv got EOF from 127.0.0.1:51012 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
W20260809 09:37:23.716588 22391 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 09:37:23.715312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51018 (local address 127.20.125.189:45207)
0809 09:37:23.715471 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0809 09:37:23.715475 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 09:37:23.715643 (+ 168us) server_negotiation.cc:407] Connection header received
0809 09:37:23.715791 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:37:23.715793 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:37:23.715844 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:37:23.715942 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0809 09:37:23.716475 (+ 533us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51018: BlockingRecv error: recv got EOF from 127.0.0.1:51018 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
I20260809 09:37:24.028131 22560 catalog_manager.cc:1259] Loaded cluster ID: bd906f270139445cbc656d95fa868aac
I20260809 09:37:24.028234 22560 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 222dee7fe32c4ccaa54c34d7a072c9da: loading cluster ID for follower catalog manager: success
I20260809 09:37:24.029932 22560 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 222dee7fe32c4ccaa54c34d7a072c9da: acquiring CA information for follower catalog manager: success
I20260809 09:37:24.031181 22560 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 222dee7fe32c4ccaa54c34d7a072c9da: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 09:37:24.051540 22587 catalog_manager.cc:1259] Loaded cluster ID: bd906f270139445cbc656d95fa868aac
I20260809 09:37:24.051630 22587 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 6aacdc8aff804d12b4457251995cc536: loading cluster ID for follower catalog manager: success
I20260809 09:37:24.053325 22587 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 6aacdc8aff804d12b4457251995cc536: acquiring CA information for follower catalog manager: success
I20260809 09:37:24.054507 22587 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 6aacdc8aff804d12b4457251995cc536: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 09:37:24.235080 22699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:37:24.230903 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51032 (local address 127.20.125.189:45207)
0809 09:37:24.231143 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0809 09:37:24.231147 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 09:37:24.231300 (+ 153us) server_negotiation.cc:407] Connection header received
0809 09:37:24.231465 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:37:24.231473 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:37:24.231601 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:37:24.231729 (+ 128us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 09:37:24.231980 (+ 251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:37:24.232571 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:37:24.233420 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:37:24.233635 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:37:24.233744 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:37:24.234055 (+ 311us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 09:37:24.234602 (+ 547us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 09:37:24.234759 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0809 09:37:24.234887 (+ 128us) server_negotiation.cc:299] Negotiation successful
0809 09:37:24.234946 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":163,"thread_start_us":87,"threads_started":1}
I20260809 09:37:24.242183 22711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:37:24.238507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37558 (local address 127.20.125.188:46547)
0809 09:37:24.238763 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0809 09:37:24.238767 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 09:37:24.238780 (+ 13us) server_negotiation.cc:407] Connection header received
0809 09:37:24.238861 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:37:24.238863 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:37:24.238979 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:37:24.239140 (+ 161us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 09:37:24.239306 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:37:24.239848 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:37:24.240629 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:37:24.240868 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:37:24.240956 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:37:24.241259 (+ 303us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 09:37:24.241677 (+ 418us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 09:37:24.241776 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0809 09:37:24.241873 (+ 97us) server_negotiation.cc:299] Negotiation successful
0809 09:37:24.241925 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":186,"thread_start_us":82,"threads_started":1}
I20260809 09:37:24.249650 22711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:37:24.245835 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37564 (local address 127.20.125.188:46547)
0809 09:37:24.245930 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0809 09:37:24.245934 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 09:37:24.245950 (+ 16us) server_negotiation.cc:407] Connection header received
0809 09:37:24.246133 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:37:24.246137 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:37:24.246193 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:37:24.246244 (+ 51us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 09:37:24.246672 (+ 428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:37:24.247412 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:37:24.248470 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:37:24.248779 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:37:24.248888 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:37:24.249168 (+ 280us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 09:37:24.249258 (+ 90us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 09:37:24.249344 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0809 09:37:24.249432 (+ 88us) server_negotiation.cc:299] Negotiation successful
0809 09:37:24.249499 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260809 09:37:24.249825 22699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:37:24.245462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51042 (local address 127.20.125.189:45207)
0809 09:37:24.245580 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0809 09:37:24.245585 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 09:37:24.245960 (+ 375us) server_negotiation.cc:407] Connection header received
0809 09:37:24.246139 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:37:24.246145 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:37:24.246202 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:37:24.246291 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 09:37:24.246522 (+ 231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:37:24.247323 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:37:24.248568 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:37:24.248893 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:37:24.248972 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:37:24.249310 (+ 338us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 09:37:24.249397 (+ 87us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 09:37:24.249462 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0809 09:37:24.249615 (+ 153us) server_negotiation.cc:299] Negotiation successful
0809 09:37:24.249673 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260809 09:37:24.250550 22718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 09:37:24.245407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32866 (local address 127.20.125.190:43161)
0809 09:37:24.245679 (+ 272us) server_negotiation.cc:206] Beginning negotiation
0809 09:37:24.245685 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 09:37:24.246012 (+ 327us) server_negotiation.cc:407] Connection header received
0809 09:37:24.246238 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 09:37:24.246241 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 09:37:24.246411 (+ 170us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 09:37:24.246573 (+ 162us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 09:37:24.246936 (+ 363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:37:24.247558 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:37:24.248698 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 09:37:24.249049 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 09:37:24.249173 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 09:37:24.249662 (+ 489us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 09:37:24.250031 (+ 369us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 09:37:24.250110 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0809 09:37:24.250310 (+ 200us) server_negotiation.cc:299] Negotiation successful
0809 09:37:24.250382 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":175,"thread_start_us":70,"threads_started":1}
I20260809 09:37:24.253880 20982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu with pid 22149
I20260809 09:37:24.267706 20982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu with pid 22268
I20260809 09:37:24.275985 20982 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYHpNFG/build/debug/bin/kudu with pid 22394
2026-08-09T09:37:24Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (3416 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (6976 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (302328 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST